Method for Non-Volatile Memory with Background Data Latch Caching During Read Operations

ABSTRACT

Part of the latency from memory read or write operations is for data to be input to or output from the data latches of the memory via an I/O bus. Methods and circuitry are present for improving performance in non-volatile memory devices by allowing the memory to perform some of these data caching and transfer operations in the background while the memory core is busy with a read operation. A scheme for caching read data is implemented so that even when the read for a current page on a current wordline must be preceded by a prerequisite read of data on an adjacent wordline, the prerequisite read along with any I/O access is preemptively done in the cycle for reading a previous page so that the current read can be performed while the previously read page is busy with the I/O access.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. application Ser. No.11/097,590 filed on Apr. 1, 2005. This application is also related tothe following U.S. patent applications: U.S. application Ser. No.11/097,517, filed Apr. 1, 2005, entitled “Use of Data Latches inMulti-Phase Programming of Non-Volatile Memories”, by Yan Li andRaul-Adrian Cernea; U.S. application Ser. No. ___/___,___, entitled“Method for Non-Volatile Memory with Background Data Latch CachingDuring Program Operations,” by Yan Li, filed concurrently herewith, onMay 5, 2006; U.S. application Ser. No.___/___,___, entitled“Non-Volatile Memory with Background Data Latch Caching During ProgramOperations,” by Yan Li, filed concurrently herewith, on May 5, 2006;U.S. application Ser. No. ___/___,___, entitled “Method for Non-VolatileMemory with Background Data Latch Caching During Erase Operations,” byJason Lin and Yan Li, filed concurrently herewith, on May 5, 2006; U.S.application Ser. No. ___/___,___, entitled “Non-Volatile Memory withBackground Data Latch Caching During Erase Operations,” by Jason Lin andYan Li, filed concurrently herewith, on May 5, 2006; U.S. applicationSer. No. ___/___,___, entitled “Non-Volatile Memory with Background DataLatch Caching During Read Operations,” by Yan Li, filed concurrentlyherewith, on May 5, 2006; U.S. application Ser. No. ___/___,___,entitled “Method for Non-Volatile Memory with Managed Execution ofCached Data,” by Yan Li, filed concurrently herewith, on May 5, 2006;and U.S. application Ser. No. ___/___,___, entitled “Non-Volatile Memorywith Managed Execution of Cached Data,” by Yan Li, filed concurrentlyherewith, on May 5, 2006. These applications are incorporated herein byreference in their entirety for all purposes.

FIELD OF THE INVENTION

This invention relates generally to non-volatile semiconductor memorysuch as electrically erasable programmable read-only memory (EEPROM) andflash EEPROM, and specifically to cache operations based on shared latchstructures allowing overlapping memory operations.

BACKGROUND OF THE INVENTION

Solid-state memory capable of nonvolatile storage of charge,particularly in the form of EEPROM and flash EEPROM packaged as a smallform factor card, has recently become the storage of choice in a varietyof mobile and handheld devices, notably information appliances andconsumer electronics products. Unlike RAM (random access memory) that isalso solid-state memory, flash memory is non-volatile, retaining itsstored data even after power is turned off. In spite of the higher cost,flash memory is increasingly being used in mass storage applications.Conventional mass storage, based on rotating magnetic medium such ashard drives and floppy disks, is unsuitable for the mobile and handheldenvironment. This is because disk drives tend to be bulky, are prone tomechanical failure and have high latency and high power requirements.These undesirable attributes make disk-based storage impractical in mostmobile and portable applications. On the other hand, flash memory, bothembedded and in the form of a removable card is ideally suited in themobile and handheld environment because of its small size, low powerconsumption, high speed and high reliability features.

EEPROM and electrically programmable read-only memory (EPROM) arenon-volatile memory that can be erased and have new data written or“programmed” into their memory cells. Both utilize a floating(unconnected) conductive gate, in a field effect transistor structure,positioned over a channel region in a semiconductor substrate, betweensource and drain regions. A control gate is then provided over thefloating gate. The threshold voltage characteristic of the transistor iscontrolled by the amount of charge that is retained on the floatinggate. That is, for a given level of charge on the floating gate, thereis a corresponding voltage (threshold) that must be applied to thecontrol gate before the transistor is turned “on” to permit conductionbetween its source and drain regions.

The floating gate can hold a range of charges and therefore can beprogrammed to any threshold voltage level within a threshold voltagewindow. The size of the threshold voltage window is delimited by theminimum and maximum threshold levels of the device, which in turncorrespond to the range of the charges that can be programmed onto thefloating gate. The threshold window generally depends on the memorydevice's characteristics, operating conditions and history. Eachdistinct, resolvable threshold voltage level range within the windowmay, in principle, be used to designate a definite memory state of thecell.

The transistor serving as a memory cell is typically programmed to a“programmed” state by one of two mechanisms. In “hot electroninjection,” a high voltage applied to the drain accelerates electronsacross the substrate channel region. At the same time a high voltageapplied to the control gate pulls the hot electrons through a thin gatedielectric onto the floating gate. In “tunneling injection,” a highvoltage is applied to the control gate relative to the substrate. Inthis way, electrons are pulled from the substrate to the interveningfloating gate.

The memory device may be erased by a number of mechanisms. For EPROM,the memory is bulk erasable by removing the charge from the floatinggate by ultraviolet radiation. For EEPROM, a memory cell is electricallyerasable, by applying a high voltage to the substrate relative to thecontrol gate so as to induce electrons in the floating gate to tunnelthrough a thin oxide to the substrate channel region (i.e.,Fowler-Nordheim tunneling.) Typically, the EEPROM is erasable byte bybyte. For flash EEPROM, the memory is electrically erasable either allat once or one or more blocks at a time, where a block may consist of512 bytes or more of memory.

Examples of Non-volatile Memory Cells

The memory devices typically comprise one or more memory chips that maybe mounted on a card. Each memory chip comprises an array of memorycells supported by peripheral circuits such as decoders and erase, writeand read circuits. The more sophisticated memory devices also come witha controller that performs intelligent and higher level memoryoperations and interfacing. There are many commercially successfulnon-volatile solid-state memory devices being used today. These memorydevices may employ different types of memory cells, each type having oneor more charge storage element.

FIGS. 1A-1E illustrate schematically different examples of non-volatilememory cells.

FIG. 1A illustrates schematically a non-volatile memory in the form ofan EEPROM cell with a floating gate for storing charge. An electricallyerasable and programmable read-only memory (EEPROM) has a similarstructure to EPROM, but additionally provides a mechanism for loadingand removing charge electrically from its floating gate upon applicationof proper voltages without the need for exposure to UV radiation.Examples of such cells and methods of manufacturing them are given inU.S. Pat. No. 5,595,924.

FIG. 1B illustrates schematically a flash EEPROM cell having both aselect gate and a control or steering gate. The memory cell 10 has a“split-channel” 12 between source 14 and drain 16 diffusions. A cell isformed effectively with two transistors T1 and T2 in series. T1 servesas a memory transistor having a floating gate 20 and a control gate 30.The floating gate is capable of storing a selectable amount of charge.The amount of current that can flow through the T1's portion of thechannel depends on the voltage on the control gate 30 and the amount ofcharge residing on the intervening floating gate 20. T2 serves as aselect transistor having a select gate 40. When T2 is turned on by avoltage at the select gate 40, it allows the current in the Ti's portionof the channel to pass between the source and drain. The selecttransistor provides a switch along the source-drain channel independentof the voltage at the control gate. One advantage is that it can be usedto turn off those cells that are still conducting at zero control gatevoltage due to their charge depletion (positive) at their floatinggates. The other advantage is that it allows source side injectionprogramming to be more easily implemented.

One simple embodiment of the split-channel memory cell is where theselect gate and the control gate are connected to the same word line asindicated schematically by a dotted line shown in FIG. 1B. This isaccomplished by having a charge storage element (floating gate)positioned over one portion of the channel and a control gate structure(which is part of a word line) positioned over the other channel portionas well as over the charge storage element. This effectively forms acell with two transistors in series, one (the memory transistor) with acombination of the amount of charge on the charge storage element andthe voltage on the word line controlling the amount of current that canflow through its portion of the channel, and the other (the selecttransistor) having the word line alone serving as its gate. Examples ofsuch cells, their uses in memory systems and methods of manufacturingthem are given in U.S. Pat. Nos. 5,070,032, 5,095,344, 5,315,541,5,343,063, and 5,661,053.

A more refined embodiment of the split-channel cell shown in FIG. 1B iswhen the select gate and the control gate are independent and notconnected by the dotted line between them. One implementation has thecontrol gates of one column in an array of cells connected to a control(or steering) line perpendicular to the word line. The effect is torelieve the word line from having to perform two functions at the sametime when reading or programming a selected cell. Those two functionsare (1) to serve as a gate of a select transistor, thus requiring aproper voltage to turn the select transistor on and off, and (2) todrive the voltage of the charge storage element to a desired levelthrough an electric field (capacitive) coupling between the word lineand the charge storage element. It is often difficult to perform both ofthese functions in an optimum manner with a single voltage. With theseparate control of the control gate and the select gate, the word lineneed only perform function (1), while the added control line performsfunction (2). This capability allows for design of higher performanceprogramming where the programming voltage is geared to the targeteddata. The use of independent control (or steering) gates in a flashEEPROM array is described, for example, in U.S. Pat. Nos. 5,313,421 and6,222,762.

FIG. 1C illustrates schematically another flash EEPROM cell having dualfloating gates and independent select and control gates. The memory cell10 is similar to that of FIG. 1B except it effectively has threetransistors in series. In this type of cell, two storage elements (i.e.,that of T1—left and T1—right) are included over its channel betweensource and drain diffusions with a select transistor T1 in between them.The memory transistors have floating gates 20 and 20′, and control gates30 and 30′, respectively. The select transistor T2 is controlled by aselect gate 40. At any one time, only one of the pair of memorytransistors is accessed for read or write. When the storage unit T1—leftis being accessed, both the T2 and T1—right are turned on to allow thecurrent in the T1—left's portion of the channel to pass between thesource and the drain. Similarly, when the storage unit T1—right is beingaccessed, T2 and T1—left are turned on. Erase is effected by having aportion of the select gate polysilicon in close proximity to thefloating gate and applying a substantial positive voltage (e.g. 20V) tothe select gate so that the electrons stored within the floating gatecan tunnel to the select gate polysilicon.

FIG. 1D illustrates schematically a string of memory cells organizedinto an NAND cell. An NAND cell 50 consists of a series of memorytransistors M1, M2, . . . Mn (n=4, 8, 16 or higher) daisy-chained bytheir sources and drains. A pair of select transistors S1, S2 controlsthe memory transistors chain's connection to the external via the NANDcell's source terminal 54 and drain terminal 56. In a memory array, whenthe source select transistor S1 is turned on, the source terminal iscoupled to a source line. Similarly, when the drain select transistor S2is turned on, the drain terminal of the NAND cell is coupled to a bitline of the memory array. Each memory transistor in the chain has acharge storage element to store a given amount of charge so as torepresent an intended memory state. A control gate of each memorytransistor provides control over read and write operations. A controlgate of each of the select transistors S1, S2 provides control access tothe NAND cell via its source terminal 54 and drain terminal 56respectively.

When an addressed memory transistor within an NAND cell is read andverified during programming, its control gate is supplied with anappropriate voltage. At the same time, the rest of the non-addressedmemory transistors in the NAND cell 50 are fully turned on byapplication of sufficient voltage on their control gates. In this way, aconductive path is effective created from the source of the individualmemory transistor to the source terminal 54 of the NAND cell andlikewise for the drain of the individual memory transistor to the drainterminal 56 of the cell. Memory devices with such NAND cell structuresare described in U.S. Pat. Nos. 5,570,315, 5,903,495, 6,046,935.

FIG. 1E illustrates schematically a non-volatile memory with adielectric layer for storing charge. Instead of the conductive floatinggate elements described earlier, a dielectric layer is used. Such memorydevices utilizing dielectric storage element have been described byEitan et al., “NROM: A Novel Localized Trapping, 2-Bit NonvolatileMemory Cell,” IEEE Electron Device Letters, vol. 21, no. 11, November2000, pp. 543-545. An ONO dielectric layer extends across the channelbetween source and drain diffusions. The charge for one data bit islocalized in the dielectric layer adjacent to the drain, and the chargefor the other data bit is localized in the dielectric layer adjacent tothe source. For example, U.S. Pat. Nos. 5,768,192 and 6,011,725 disclosea nonvolatile memory cell having a trapping dielectric sandwichedbetween two silicon dioxide layers. Multi-state data storage isimplemented by separately reading the binary states of the spatiallyseparated charge storage regions within the dielectric.

Memory Array

A memory device typically comprises of a two-dimensional array of memorycells arranged in rows and columns and addressable by word lines and bitlines. The array can be formed according to an NOR type or an NAND typearchitecture.

NOR Array

FIG. 2 illustrates an example of an NOR array of memory cells. Memorydevices with an NOR type architecture have been implemented with cellsof the type illustrated in FIGS. 1B or 1C. Each row of memory cells areconnected by their sources and drains in a daisy-chain manner. Thisdesign is sometimes referred to as a virtual ground design. Each memorycell 10 has a source 14, a drain 16, a control gate 30 and a select gate40. The cells in a row have their select gates connected to word line42. The cells in a column have their sources and drains respectivelyconnected to selected bit lines 34 and 36. In some embodiments where thememory cells have their control gate and select gate controlledindependently, a steering line 36 also connects the control gates of thecells in a column.

Many flash EEPROM devices are implemented with memory cells where eachis formed with its control gate and select gate connected together. Inthis case, there is no need for steering lines and a word line simplyconnects all the control gates and select gates of cells along each row.Examples of these designs are disclosed in U.S. Pat. Nos. 5,172,338 and5,418,752. In these designs, the word line essentially performed twofunctions: row selection and supplying control gate voltage to all cellsin the row for reading or programming.

NAND Array

FIG. 3 illustrates an example of an NAND array of memory cells, such asthat shown in FIG. 1D. Along each column of NAND cells, a bit line iscoupled to the drain terminal 56 of each NAND cell. Along each row ofNAND cells, a source line may connect all their source terminals 54.Also the control gates of the NAND cells along a row are connected to aseries of corresponding word lines. An entire row of NAND cells can beaddressed by turning on the pair of select transistors (see FIG. 1D)with appropriate voltages on their control gates via the connected wordlines. When a memory transistor within the chain of a NAND cell is beingread, the remaining memory transistors in the chain are turned on hardvia their associated word lines so that the current flowing through thechain is essentially dependent upon the level of charge stored in thecell being read. An example of an NAND architecture array and itsoperation as part of a memory system is found in U.S. Pat. Nos.5,570,315, 5,774,397 and 6,046,935.

Block Erase

Programming of charge storage memory devices can only result in addingmore charge to its charge storage elements. Therefore, prior to aprogram operation, existing charge in a charge storage element must beremoved (or erased). Erase circuits (not shown) are provided to eraseone or more blocks of memory cells. A non-volatile memory such as EEPROMis referred to as a “Flash” EEPROM when an entire array of cells, orsignificant groups of cells of the array, is electrically erasedtogether (i.e., in a flash). Once erased, the group of cells can then bereprogrammed. The group of cells erasable together may consist one ormore addressable erase unit. The erase unit or block typically storesone or more pages of data, the page being the unit of programming andreading, although more than one page may be programmed or read in asingle operation. Each page typically stores one or more sectors ofdata, the size of the sector being defined by the host system. Anexample is a sector of 512 bytes of user data, following a standardestablished with magnetic disk drives, plus some number of bytes ofoverhead information about the user data and/or the block in with it isstored.

Read/Write Circuits

In the usual two-state EEPROM cell, at least one current breakpointlevel is established so as to partition the conduction window into tworegions. When a cell is read by applying predetermined, fixed voltages,its source/drain current is resolved into a memory state by comparingwith the breakpoint level (or reference current I_(REF)). If the currentread is higher than that of the breakpoint level, the cell is determinedto be in one logical state (e.g., a “zero” state). On the other hand, ifthe current is less than that of the breakpoint level, the cell isdetermined to be in the other logical state (e.g., a “one” state). Thus,such a two-state cell stores one bit of digital information. A referencecurrent source, which may be externally programmable, is often providedas part of a memory system to generate the breakpoint level current.

In order to increase memory capacity, flash EEPROM devices are beingfabricated with higher and higher density as the state of thesemiconductor technology advances. Another method for increasing storagecapacity is to have each memory cell store more than two states.

For a multi-state or multi-level EEPROM memory cell, the conductionwindow is partitioned into more than two regions by more than onebreakpoint such that each cell is capable of storing more than one bitof data. The information that a given EEPROM array can store is thusincreased with the number of states that each cell can store. EEPROM orflash EEPROM with multi-state or multi-level memory cells have beendescribed in U.S. Pat. No. 5,172,338.

In practice, the memory state of a cell is usually read by sensing theconduction current across the source and drain electrodes of the cellwhen a reference voltage is applied to the control gate. Thus, for eachgiven charge on the floating gate of a cell, a corresponding conductioncurrent with respect to a fixed reference control gate voltage may bedetected. Similarly, the range of charge programmable onto the floatinggate defines a corresponding threshold voltage window or a correspondingconduction current window.

Alternatively, instead of detecting the conduction current among apartitioned current window, it is possible to set the threshold voltagefor a given memory state under test at the control gate and detect ifthe conduction current is lower or higher than a threshold current. Inone implementation the detection of the conduction current relative to athreshold current is accomplished by examining the rate the conductioncurrent is discharging through the capacitance of the bit line.

FIG. 4 illustrates the relation between the source-drain current I_(D)and the control gate voltage V_(CG) for four different charges Q1-Q4that the floating gate may be selectively storing at any one time. Thefour solid I_(D) versus V_(CG) curves represent four possible chargelevels that can be programmed on a floating gate of a memory cell,respectively corresponding to four possible memory states. As anexample, the threshold voltage window of a population of cells may rangefrom 0.5V to 3.5V. Six memory states may be demarcated by partitioningthe threshold window into five regions in interval of 0.5V each. Forexample, if a reference current, I_(REF) of 2 μA is used as shown, thenthe cell programmed with Q1 may be considered to be in a memory state“1” since its curve intersects with I_(REF) in the region of thethreshold window demarcated by V_(CG)=0.5V and 1.0V. Similarly, Q4 is ina memory state “5”.

As can be seen from the description above, the more states a memory cellis made to store, the more finely divided is its threshold window. Thiswill require higher precision in programming and reading operations inorder to be able to achieve the required resolution.

U.S. Pat. No. 4,357,685 discloses a method of programming a 2-stateEPROM in which when a cell is programmed to a given state, it is subjectto successive programming voltage pulses, each time adding incrementalcharge to the floating gate. In between pulses, the cell is read back orverified to determine its source-drain current relative to thebreakpoint level. Programming stops when the current state has beenverified to reach the desired state. The programming pulse train usedmay have increasing period or amplitude.

Prior art programming circuits simply apply programming pulses to stepthrough the threshold window from the erased or ground state until thetarget state is reached. Practically, to allow for adequate resolution,each partitioned or demarcated region would require at least about fiveprogramming steps to transverse. The performance is acceptable for2-state memory cells. However, for multi-state cells, the number ofsteps required increases with the number of partitions and therefore,the programming precision or resolution must be increased. For example,a 16-state cell may require on average at least 40 programming pulses toprogram to a target state.

FIG. 5 illustrates schematically a memory device with a typicalarrangement of a memory array 100 accessible by read/write circuits 170via row decoder 130 and column decoder 160. As described in connectionwith FIGS. 2 and 3, a memory transistor of a memory cell in the memoryarray 100 is addressable via a set of selected word line(s) and bitline(s). The row decoder 130 selects one or more word lines and thecolumn decoder 160 selects one or more bit lines in order to applyappropriate voltages to the respective gates of the addressed memorytransistor. Read/write circuits 170 are provided to read or write(program) the memory states of addressed memory transistors. Theread/write circuits 170 comprise a number of read/write modulesconnectable via bit lines to memory elements in the array.

FIG. 6A is a schematic block diagram of an individual read/write module190. Essentially, during read or verify, a sense amplifier determinesthe current flowing through the drain of an addressed memory transistorconnected via a selected bit line. The current depends on the chargestored in the memory transistor and its control gate voltage. Forexample, in a multi-state EEPROM cell, its floating gate can be chargedto one of several different levels. For a 4-level cell, it may be usedto store two bits of data. The level detected by the sense amplifier isconverted by a level-to-bits conversion logic to a set of data bits tobe stored in a data latch.

Factors Affecting Read/Write Performance and Accuracy

In order to improve read and program performance, multiple chargestorage elements or memory transistors in an array are read orprogrammed in parallel. Thus, a logical “page” of memory elements areread or programmed together. In existing memory architectures, a rowtypically contains several interleaved pages. All memory elements of apage will be read or programmed together. The column decoder willselectively connect each one of the interleaved pages to a correspondingnumber of read/write modules. For example, in one implementation, thememory array is designed to have a page size of 532 bytes (512 bytesplus 20 bytes of overheads.) If each column contains a drain bit lineand there are two interleaved pages per row, this amounts to 8512columns with each page being associated with 4256 columns. There will be4256 sense modules connectable to read or write in parallel either allthe even bit lines or the odd bit lines. In this way, a page of 4256bits (i.e., 532 bytes) of data in parallel are read from or programmedinto the page of memory elements. The read/write modules forming theread/write circuits 170 can be arranged into various architectures.

Referring to FIG. 5, the read/write circuits 170 is organized into banksof read/write stacks 180. Each read/write stack 180 is a stack ofread/write modules 190. In a memory array, the column spacing isdetermined by the size of the one or two transistors that occupy it.However, as can be seen from FIG. 6A, the circuitry of a read/writemodule will likely be implemented with many more transistors and circuitelements and therefore will occupy a space over many columns. In orderto service more than one column among the occupied columns, multiplemodules are stacked up on top of each other.

FIG. 6B shows the read/write stack of FIG. 5 implemented conventionallyby a stack of read/write modules 190. For example, a read/write modulemay extend over sixteen columns, then a read/write stack 180 with astack of eight read/write modules can be used to service eight columnsin parallel. The read/write stack can be coupled via a column decoder toeither the eight odd (1, 3, 5, 7, 9, 11, 13, 15) columns or the eighteven (2, 4, 6, 8, 10, 12, 14, 16) columns among the bank.

As mentioned before, conventional memory devices improve read/writeoperations by operating in a massively parallel manner on all even orall odd bit lines at a time. This architecture of a row consisting oftwo interleaved pages will help to alleviate the problem of fitting theblock of read/write circuits. It is also dictated by consideration ofcontrolling bit-line to bit-line capacitive coupling. A block decoder isused to multiplex the set of read/write modules to either the even pageor the odd page. In this way, whenever one set bit lines are being reador programmed, the interleaving set can be grounded to minimizeimmediate neighbor coupling.

However, the interleaving page architecture is disadvantageous in atleast three respects. First, it requires additional multiplexingcircuitry. Secondly, it is slow in performance. To finish read orprogram of memory cells connected by a word line or in a row, two reador two program operations are required. Thirdly, it is also not optimumin addressing other disturb effects such as field coupling betweenneighboring charge storage elements at the floating gate level when thetwo neighbors are programmed at different times, such as separately inodd and even pages.

The problem of neighboring field coupling becomes more pronounced withever closer spacing between memory transistors. In a memory transistor,a charge storage element is sandwiched between a channel region and acontrol gate. The current that flows in the channel region is a functionof the resultant electric field contributed by the field at the controlgate and the charge storage element. With ever increasing density,memory transistors are formed closer and closer together. The field fromneighboring charge elements then becomes significant contributor to theresultant field of an affected cell. The neighboring field depends onthe charge programmed into the charge storage elements of the neighbors.This perturbing field is dynamic in nature as it changes with theprogrammed states of the neighbors. Thus, an affected cell may readdifferently at different time depending on the changing states of theneighbors.

The conventional architecture of interleaving page exacerbates the errorcaused by neighboring floating gate coupling. Since the even page andthe odd page are programmed and read independently of each other, a pagemay be programmed under one set of condition but read back under anentirely different set of condition, depending on what has happened tothe intervening page in the meantime. The read errors will become moresevere with increasing density, requiring a more accurate read operationand coarser partitioning of the threshold window for multi-stateimplementation. Performance will suffer and the potential capacity in amulti-state implementation is limited.

United States Patent Publication No. US-2004-0060031-A1 discloses a highperformance yet compact non-volatile memory device having a large blockof read/write circuits to read and write a corresponding block of memorycells in parallel. In particular, the memory device has an architecturethat reduces redundancy in the block of read/write circuits to aminimum. Significant saving in space as well as power is accomplished byredistributing the block of read/write modules into a block read/writemodule core portions that operate in parallel while interacting with asubstantially smaller sets of common portions in a time-multiplexingmanner. In particular, data processing among read/write circuits betweena plurality of sense amplifiers and data latches is performed by ashared processor.

Therefore there is a general need for high performance and high capacitynon-volatile memory. In particular, there is a need for a compactnon-volatile memory with enhanced read and program performance having animproved processor that is compact and efficient, yet highly versatilefor processing data among the read/writing circuits.

SUMMARY OF INVENTION

According to one aspect of the invention, cache operations are presentedthat allow data to be transferred in or out of a memory while theinternal memory is engaged in another operation, such as a read, programor erase. In particular, arrangements of data latches and methods oftheir use are described which allow such cache operations.

Architectures are described where data latches are shared by a number ofphysical pages. For example, read/write stacks are associated with thebit lines of the memory, which shared by multiple word lines. While oneoperation is going on in the memory, if any of these latch are free,they can cache data for future operations in the same or another wordline, saving transfer time as this can be hidden behind anotheroperation. This can improve performance by increasing the amount ofpipelining of different operations or phases of operations. In oneexample, in a cache program operation, while programming one page ofdata another page of data can be loaded in, saving on transfer time. Foranother example, in one exemplary embodiment, a read operation on oneword line is inserted into a write operation on another word line,allowing the data from the read to be transferred out of the memorywhile the data write continues on.

According to the various aspects, data from another page in the sameblock, but on a different word line, can be toggled out (to, forexample, do an ECC operation) while a write or other operation is goingon for the first page of data. This inter-phase pipelining of operationsallows the time needed for the data transfer to be hidden behind theoperation on the first page of data. More generally, this allows aportion of one operation to be inserted between phases of another,typically longer, operation. Another example would be to insert asensing operation between phases of, say, an erase operation, such asbefore an erase pulse or before a soft programming phase used as thelater part of the erase.

If a relatively long operation with different phases is being performed,a primary aspect will interpose in a quicker operation using the sharedlatches of the read/write stacks if latches available. For example, aread can be inserted into a program or erase operation, or a binaryprogram can be inserted into an erase. The primary exemplary embodimentswill toggle data in and/or out for one page during a program operationfor another page that shares the same read write stacks, where, forexample, a read of the data to be toggled out and modified is insertedinto the verify phase of the data write.

The availability of open data latches can arise in a number of ways.Generally, for a memory storing n bits per cell, n such data latcheswill be needed for each bit line; however, not all of these latches areneeded at all times. For example, in a two-bit per cell memory storingdata in an upper page/lower page format, one data latches will be neededwhile programming the lower page (with another latch used if quick passwrite is implemented). Two data latches will be needed while programmingthe upper page (with a third latch used if quick pass write isimplemented)). More generally, for memories storing multiple pages, allof the latches will be needed only when programming the highest page.This leaves the other latches available for cache operations. Further,even while writing the highest page, as the various states are removedfrom the verify phase of the write operation, latches will free up.Specifically, once only the highest state remains to be verified, only asingle latch is needed for verification purposes and the others may beused for cache operations.

An exemplary embodiment is based on a four state memory storing two-bitsper cell and having two latches for data on each bit line and oneadditional latch for quick pass write. The operations of writing thelower page, or erasing, or doing a post erase soft program are basicallya binary operation and have one of the data latches free, which can useit to cache data. Similarly, where doing an upper page or full sequencewrite, once all but the highest level has verified, only a single stateneeds to verify and the memory can free up a latch that can be used tocache data. An example of how this can be used is that when programmingone page, such as in a copy operation, a read of another page thatshares the same set of data latches, such as another word line on thesame set of bit lines, can be slipped in between program pulse andverifies of the write. The address can then be switched to the pagebeing written, allowing the write process to pick up where it left offwithout having to restart. While the write continues, the data cachedduring the interpolated read can be toggled out, checked or modified andtransferred back to be present for writing back in once the earlierwrite operation completes. This sort cache operation allows the togglingout and modification of the second page of data to be hidden behind theprogramming of the first page.

Cache Operations During Read—Background Read and Write Operations

According to one aspect of the invention, a read cache scheme isprovided for the case of multi-state memory cells with the need tominimize perturbation between the memory cells (Yupin Effect.) Thescheme for caching read data is such that even for read operation whosecorrection depend on data from a neighboring physical page or wordline,the data latches and I/O bus are efficiently used to toggle out apreviously read page while a current page is being sensed from thememory core.

In a preferred implementation, the efficient read caching scheme isemployed for memory encoded with the “LM” coding and read withlook-ahead (“LA”) correction. Both the “LM” coding and “LA” correctionrequire additional latch and bus activities besides the mere toggling ofread data. When the read for a current page on a current wordline mustbe preceded by a prerequisite read of data on an adjacent wordline, theprerequisite read along with any I/O access is preemptively done in thecycle for reading a previous page so that the current read can beperformed while the previously read page is busy with the I/O access.

According to another aspect of the invention, a read caching scheme isimplemented for multi-bit pages of memory cells where more than one bitis sensed together. In the preferred implementation, all of themulti-bits are sensed together. The multi-bit pages of memory cellssensed correspond to multiple binary pages, one for each of themulti-bits. The binary pages are outputting in each cycle, while inmultiple of cycles the multi-bit sensing is performed. The multiplebeing the same as the number of bits in the multi-bits.

Additional features and advantages of the present invention will beunderstood from the following description of its preferred embodiments,which description should be taken in conjunction with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1E illustrate schematically different examples of non-volatilememory cells.

FIG. 2 illustrates an example of an NOR array of memory cells.

FIG. 3 illustrates an example of an NAND array of memory cells, such asthat shown in FIG. 1D.

FIG. 4 illustrates the relation between the source-drain current and thecontrol gate voltage for four different charges Q1-Q4 that the floatinggate may be storing at any one time.

FIG. 5 illustrates schematically a typical arrangement of a memory arrayaccessible by read/write circuits via row and column decoders.

FIG. 6A is a schematic block diagram of an individual read/write module.

FIG. 6B shows the read/write stack of FIG. 5 implemented conventionallyby a stack of read/write modules.

FIG. 7A illustrates schematically a compact memory device having a bankof partitioned read/write stacks, in which the improved processor of thepresent invention is implemented.

FIG. 7B illustrates a preferred arrangement of the compact memory deviceshown in FIG. 7A.

FIG. 8 illustrates schematically a general arrangement of the basiccomponents in a read/write stack shown in FIG. 7A.

FIG. 9 illustrates one preferred arrangement of the read/write stacksamong the read/write circuits shown in FIGS. 7A and 7B.

FIG. 10 illustrates an improved embodiment of the common processor shownin FIG. 9.

FIG. 11A illustrates a preferred embodiment of the input logic of thecommon processor shown in FIG. 10.

FIG. 11B illustrates the truth table of the input logic of FIG. 11A.

FIG. 12A illustrates a preferred embodiment of the output logic of thecommon processor shown in FIG. 10.

FIG. 12B illustrates the truth table of the output logic of FIG. 12A.

FIG. 13 is a simplified version of FIG. 10 that shows some specificelements that are relevant to the present discussion in a two-bitembodiment of the present invention

FIG. 14 indicates the latch assignment for the same elements as FIG. 13for upper page program where the lower page data is read in.

FIG. 15 illustrates aspects of cache program in the single page mode.

FIG. 16 shows a programming waveform that can be used in a lower page tofull sequence conversion.

FIG. 17 illustrates the relative timing in a cache program operationwith a full sequence conversion.

FIG. 18 describes the disposition of latches in a cache page copyoperation.

FIGS. 19A and 19B illustrate the relative timings in cache page copyoperations.

FIG. 20A illustrates threshold voltage distributions of the 4-statememory array when each memory cell stores two bits of data using the LMcode.

FIG. 20B illustrates the lower page programming in an existing, 2-roundprogramming scheme using the LM code.

FIG. 20C illustrates the upper page programming in an existing, 2-roundprogramming scheme using the LM code.

FIG. 20D illustrates the read operation that is required to discern thelower bit of the 4-state memory encoded with the LM code.

FIG. 20E illustrates the read operation that is required to discern theupper bit of the 4-state memory encoded with the LM code.

FIG. 21 is a schematic timing diagram for a lower page programming,illustrating background operation of loading a next page of program datainto unused data latches.

FIG. 22 is a table showing the number of states that needs to be trackedduring various phases of a 4-state upper page or full sequenceprogramming employing QWP.

FIG. 23 is a schematic timing diagram for an upper page or full sequenceprogramming, illustrating background operation of loading a next page ofprogram data into unused data latches.

FIG. 24 is a flowchart illustrating latch operations contemporaneouswith a current multi-phase memory operation, according to a generalembodiment of the invention.

FIG. 25 is a schematic timing diagram for a lower page programming,illustrating a read interrupt operation using available latches.

FIG. 26 is a schematic timing diagram for an upper page programming,illustrating a read interrupt operation using available latches.

FIG. 27 illustrates the package of information associated with a typicalmemory operation.

FIG. 28 illustrates a conventional memory system that supports simplecache operations.

FIG. 29 is a flow diagram illustrating the queuing and possible mergingof multiple memory operations.

FIG. 30 illustrates a schematic block diagram of a preferred on-chipcontrol circuitry incorporating a memory operation queue and a memoryoperation queue manager.

FIG. 31 is a schematic flow diagram illustrating a cache operation inthe background during an erase operation.

FIG. 32 is a schematic timing diagram for an erase operation on thememory array, illustrating a program data loading operation during thefirst, erase phase of the erase operation.

FIG. 33 is a schematic timing diagram for an erase operation on thememory array, illustrating a program data loading operation during thesoft programming/verifying phase of the erase operation.

FIG. 34 is a schematic timing diagram for an erase operation on thememory array, illustrating a read operation being inserted and theresulting data output operation using available latches.

FIG. 35 is a schematic flow diagram illustrating a specific cacheoperation for read scrub application in the background during an eraseoperation in STEP 780 of FIG. 31.

FIG. 36 illustrates a preemptive background read during erase.

FIG. 37 illustrates schematically a typical read cache scheme.

FIG. 38A is a schematic timing diagram for cache reading a logical pageencoded with the LM code.

FIG. 38B is a schematic timing diagram for cache reading with LM code inthe special case of reading a lower-bit logical page when the upper-bitlogical page has not yet been programmed.

FIG. 39 illustrates a schematic timing diagram for cache read withall-bit sensing for a 2-bit memory.

FIG. 40 illustrates an example of a memory having 2-bit memory cells andwith its pages programmed in an optimal sequence so as to minimize theYupin Effect between memory cells on adjacent wordlines.

FIG. 41 illustrates an implementation of read caching for the LM codewith LA correction according to the convention scheme shown in FIG. 37.

FIG. 42 illustrates an improved read caching scheme with the LM code andLA correction.

FIG. 43 is a schematic flow diagram illustrating the improved readcaching.

FIG. 44 is a schematic flow diagram illustrating a further articulationof STEP 850 of FIG. 43.

FIG. 45 is a schematic flow diagram illustrating a further articulationof STEP 830 of FIG. 43.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 7A illustrates schematically a compact memory device having a bankof partitioned read/write stacks, in which the improved processor of thepresent invention is implemented. The memory device includes atwo-dimensional array of memory cells 300, control circuitry 310, andread/write circuits 370. The memory array 300 is addressable by wordlines via a row decoder 330 and by bit lines via a column decoder 360.The read/write circuits 370 is implemented as a bank of partitionedread/write stacks 400 and allows a block (also referred to as a “page”)of memory cells to be read or programmed in parallel. In a preferredembodiment, a page is constituted from a contiguous row of memory cells.In another embodiment, where a row of memory cells are partitioned intomultiple blocks or pages, a block multiplexer 350 is provided tomultiplex the read/write circuits 370 to the individual blocks.

The control circuitry 310 cooperates with the read/write circuits 370 toperform memory operations on the memory array 300. The control circuitry310 includes a state machine 312, an on-chip address decoder 314 and apower control module 316. The state machine 312 provides chip levelcontrol of memory operations. The on-chip address decoder 314 providesan address interface between that used by the host or a memorycontroller to the hardware address used by the decoders 330 and 370. Thepower control module 316 controls the power and voltages supplied to theword lines and bit lines during memory operations.

FIG. 7B illustrates a preferred arrangement of the compact memory deviceshown in FIG. 7A. Access to the memory array 300 by the variousperipheral circuits is implemented in a symmetric fashion, on oppositesides of the array so that access lines and circuitry on each side arereduced in half. Thus, the row decoder is split into row decoders 330Aand 330B and the column decoder into column decoders 360A and 360B. Inthe embodiment where a row of memory cells are partitioned into multipleblocks, the block multiplexer 350 is split into block multiplexers 350Aand 350B. Similarly, the read/write circuits are split into read/writecircuits 370A connecting to bit lines from the bottom and read/writecircuits 370B connecting to bit lines from the top of the array 300. Inthis way, the density of the read/write modules, and therefore that ofthe partitioned read/write stacks 400, is essentially reduced by onehalf.

FIG. 8 illustrates schematically a general arrangement of the basiccomponents in a read/write stack shown in FIG. 7A. According to ageneral architecture of the invention, the read/write stack 400comprises a stack of sense amplifiers 212 for sensing k bit lines, anI/O module 440 for input or output of data via an I/O bus 231, a stackof data latches 430 for storing input or output data, a common processor500 to process and store data among the read/write stack 400, and astack bus 421 for communication among the stack components. A stack buscontroller among the read/write circuits 370 provides control and timingsignals via lines 411 for controlling the various components among theread/write stacks.

FIG. 9 illustrates one preferred arrangement of the read/write stacksamong the read/write circuits shown in FIGS. 7A and 7B. Each read/writestack 400 operates on a group of k bit lines in parallel. If a page hasp=r*k bit lines, there will be r read/write stacks, 400-1, . . . ,400-r.

The entire bank of partitioned read/write stacks 400 operating inparallel allows a block (or page) of p cells along a row to be read orprogrammed in parallel. Thus, there will be p read/write modules for theentire row of cells. As each stack is serving k memory cells, the totalnumber of read/write stacks in the bank is therefore given by r=p/k. Forexample, if r is the number of stacks in the bank, then p=r*k. Oneexample memory array may have p=512 bytes (512×8 bits), k=8, andtherefore r=512. In the preferred embodiment, the block is a run of theentire row of cells. In another embodiment, the block is a subset ofcells in the row. For example, the subset of cells could be one half ofthe entire row or one quarter of the entire row. The subset of cellscould be a run of contiguous cells or one every other cell, or one everypredetermined number of cells.

Each read/write stack, such as 400-1, essentially contains a stack ofsense amplifiers 212-1 to 212-k servicing a segment of k memory cells inparallel. A preferred sense amplifier is disclosed in United StatesPatent Publication No. 2004-0109357-A1, the entire disclosure of whichis hereby incorporated herein by reference.

The stack bus controller 410 provides control and timing signals to theread/write circuit 370 via lines 411. The stack bus controller is itselfdependent on the memory controller 310 via lines 311. Communicationamong each read/write stack 400 is effected by an interconnecting stackbus 431 and controlled by the stack bus controller 410. Control lines411 provide control and clock signals from the stack bus controller 410to the components of the read/write stacks 400-1.

In the preferred arrangement, the stack bus is partitioned into a SABus422 for communication between the common processor 500 and the stack ofsense amplifiers 212, and a DBus 423 for communication between theprocessor and the stack of data latches 430.

The stack of data latches 430 comprises of data latches 430-1 to 430-k,one for each memory cell associated with the stack The I/O module 440enables the data latches to exchange data with the external via an I/Obus 231.

The common processor also includes an output 507 for output of a statussignal indicating a status of the memory operation, such as an errorcondition. The status signal is used to drive the gate of ann-transistor 550 that is tied to a FLAG BUS 509 in a Wired-Orconfiguration. The FLAG BUS is preferably precharged by the controller310 and will be pulled down when a status signal is asserted by any ofthe read/write stacks.

FIG. 10 illustrates an improved embodiment of the common processor shownin FIG. 9. The common processor 500 comprises a processor bus, PBUS 505for communication with external circuits, an input logic 510, aprocessor latch PLatch 520 and an output logic 530.

The input logic 510 receives data from the PBUS and outputs to a BSInode as a transformed data in one of logical states “1”, “0”, or “Z”(float) depending on the control signals from the stack bus controller410 via signal lines 411. A Set/Reset latch, PLatch 520 then latchesBSI, resulting in a pair of complementary output signals as MTCH andMTCH*.

The output logic 530 receives the MTCH and MTCH* signals and outputs onthe PBUS 505 a transformed data in one of logical states “1”, “0”, or“Z” (float) depending on the control signals from the stack buscontroller 410 via signal lines 411.

At any one time the common processor 500 processes the data related to agiven memory cell. For example, FIG. 10 illustrates the case for thememory cell coupled to bit line 1. The corresponding sense amplifier212-1 comprises a node where the sense amplifier data appears. In thepreferred embodiment, the node assumes the form of a SA Latch, 214-1that stores data. Similarly, the corresponding set of data latches 430-1stores input or output data associated with the memory cell coupled tobit line 1. In the preferred embodiment, the set of data latches 430-1comprises sufficient data latches, 434-1, . . . , 434-n for storingn-bits of data.

The PBUS 505 of the common processor 500 has access to the SA latch214-1 via the SBUS 422 when a transfer gate 501 is enabled by a pair ofcomplementary signals SAP and SAN. Similarly, the PBUS 505 has access tothe set of data latches 430-1 via the DBUS 423 when a transfer gate 502is enabled by a pair of complementary signals DTP and DTN. The signalsSAP, SAN, DTP and DTN are illustrated explicitly as part of the controlsignals from the stack bus controller 410.

FIG. 11A illustrates a preferred embodiment of the input logic of thecommon processor shown in FIG. 10. The input logic 520 receives the dataon the PBUS 505 and depending on the control signals, either has theoutput BSI being the same, or inverted, or floated. The output BSI nodeis essentially affected by either the output of a transfer gate 522 or apull-up circuit comprising p-transistors 524 and 525 in series to Vdd,or a pull-down circuit comprising n-transistors 526 and 527 in series toground. The pull-up circuit has the gates to the p-transistor 524 and525 respectively controlled by the signals PBUS and ONE. The pull-downcircuit has the gates to the n-transistors 526 and 527 respectivelycontrolled by the signals ONEB<1> and PBUS.

FIG. 11B illustrates the truth table of the input logic of FIG. 11A. Thelogic is controlled by PBUS and the control signals ONE, ONEB<0>,ONEB<1> which are part of the control signals from the stack buscontroller 410. Essentially, three transfer modes, PASSTHROUGH,INVERTED, and FLOATED, are supported.

In the case of the PASSTHROUGH mode where BSI is the same as the inputdata, the signals ONE is at a logical “1”, ONEB<0> at “0” and ONEB<1> at“0”. This will disable the pull-up or pull-down but enable the transfergate 522 to pass the data on the PBUS 505 to the output 523. In the caseof the INVERTED mode where BSI is the invert of the input data, thesignals ONE is at “0”, ONEB<0> at “1” and ONE<1> at “1”. This willdisable the transfer gate 522. Also, when PBUS is at “0”, the pull-downcircuit will be disabled while the pull-up circuit is enabled, resultingin BSI being at “1”. Similarly, when PBUS is at “1”, the pull-up circuitis disabled while the pull-down circuit is enabled, resulting in BSIbeing at “0”. Finally, in the case of the FLOATED mode, the output BSIcan be floated by having the signals ONE at “1”, ONEB<0> at “1” andONEB<1> at “0”. The FLOATED mode is listed for completeness although inpractice, it is not used.

FIG. 12A illustrates a preferred embodiment of the output logic of thecommon processor shown in FIG. 10. The signal at the BSI node from theinput logic 520 is latched in the processor latch, PLatch 520. Theoutput logic 530 receives the data MTCH and MTCH* from the output ofPLatch 520 and depending on the control signals, outputs on the PBUS aseither in a PASSTHROUGH, INVERTED OR FLOATED mode. In other words, thefour branches act as drivers for the PBUS 505, actively pulling iteither to a HIGH, LOW or FLOATED state. This is accomplished by fourbranch circuits, namely two pull-up and two pull-down circuits for thePBUS 505. A first pull-up circuit comprises p-transistors 531 and 532 inseries to Vdd, and is able to pull up the PBUS when MTCH is at “0”. Asecond pull-up circuit comprises p-transistors 533 and 534 in series toground and is able to pull up the PBUS when MTCH is at “1”. Similarly, afirst pull-down circuit comprises n-transistors 535 and 536 in series toVdd, and is able to pull down the PBUS when MTCH is at “0”. A secondpull-up circuit comprises n-transistors 537 and 538 in series to groundand is able to pull up the PBUS when MTCH is at “1”.

One feature of the invention is to constitute the pull-up circuits withPMOS transistors and the pull-down circuits with NMOS transistors. Sincethe pull by the NMOS is much stronger than that of the PMOS, thepull-down will always overcome the pull-up in any contentions. In otherwords, the node or bus can always default to a pull-up or “1” state, andif desired, can always be flipped to a “0” state by a pull-down.

FIG. 12B illustrates the truth table of the output logic of FIG. 12A.The logic is controlled by MTCH, MTCH* latched from the input logic andthe control signals PDIR, PINV, NDIR, NINV, which are part of thecontrol signals from the stack bus controller 410. Four operation modes,PASSTHROUGH, INVERTED, FLOATED, and PRECHARGE are supported.

In the FLOATED mode, all four branches are disabled. This isaccomplished by having the signals PINV=1, NINV=0, PDIR=1, NDIR=0, whichare also the default values. In the PASSTHROUGH mode, when MTCH=0, itwill require PBUS=0. This is accomplished by only enabling the pull-downbranch with n-transistors 535 and 536, with all control signals at theirdefault values except for NDIR=1. When MTCH=1, it will require PBUS=1.This is accomplished by only enabling the pull-up branch withp-transistors 533 and 534, with all control signals at their defaultvalues except for PINV=0. In the INVERTED mode, when MTCH=0, it willrequire PBUS=1. This is accomplished by only enabling the pull-up branchwith p-transistors 531 and 532, with all control signals at theirdefault values except for PDIR=0. When MTCH=1, it will require PBUS=0.This is accomplished by only enabling the pull-down branch withn-transistors 537 and 538, with all control signals at their defaultvalues except for NINV=1. In the PRECHARGE mode, the control signalssettings of PDIR=0 and PINV=0 will either enable the pull-up branch withp-transistors 531 and 531 when MTCH=1 or the pull-up branch withp-transistors 533 and 534 when MTCH=0.

Common processor operations are developed more fully in U.S. patentapplication Ser. No. 11/026,536, Dec. 29, 2004, which is herebyincorporated in its entirety by this reference.

Use of Data Latches in Cache Operations

A number of aspects of the present invention make use of the datalatches of the read/write stacks described above in FIG. 10 for cacheoperations that will data in and out while the internal memory is doingother operations such as read, write, or erase. In the above-describedarchitectures, data latches are shared by a number of physical pages.For example, as on the read/write stacks of the bit lines, shared by allof the word lines, so while one operation is going on, if any of theselatches are free, they can cache data for future operations in the sameor another word line, saving transfer time as this can be hidden behindanother operation. This can improve performance by increasing the amountof pipelining of different operations or phases of operations. In oneexample, in a cache program operation, while programming one page ofdata another page of data can be loaded in, saving on transfer time. Foranother example, in one exemplary embodiment, a read operation on oneword line is inserted into a write operation on another word line,allowing the data from the read to be transferred out of the memorywhile the data write continues on.

Note that this allows data from another page in the same block, but on adifferent word line, to be toggled out (to, for example, do an ECCoperation) while the write or other operation is going on for the firstpage of data. This inter-phase pipelining of operations allows the timeneeded for the data transfer to be hidden behind the operation on thefirst page of data. More generally, this allows a portion of oneoperation to be inserted between phases of another, typically longer,operation. Another example would be to insert a sensing operationbetween phases of, say, an erase operation, such as before an erasepulse or before a soft programming phase used as the later part of theerase.

To make the relative times needed for some of the operations discussed,a set of exemplary time values for the system described above can betake as:

-   -   Data write: ˜700 μs (lower page˜600 μs, upper page 800 μs)    -   Binary data write: ˜200 μs    -   Erase: ˜2,500 μs    -   Read: ˜20-40 μs    -   Read and toggle out data: 2 KB data, ˜80 μs; 4 KB˜160 μs; 8        KB˜320 μs

These values can be used for reference to give an idea of the relativetimes involved for the timing diagrams below. If have a long operationwith different phases, a primary aspect will interpose in a quickeroperation using the shared latches of the read/write stacks if latchesavailable. For example, a read can be inserted into a program or eraseoperation, or a binary program can be inserted into an erase. Theprimary exemplary embodiments will toggle data in and/or out for onepage during a program operation for another page that shares the sameread write stacks, where, for example, a read of the data to be toggledout and modified is inserted into the verify phase of the data write.

The availability of open data latches can arise in a number of ways.Generally, for a memory storing n bits per cell, n such data latcheswill be needed for each bit line; however, not all of these latches areneeded at all times. For example, in a two-bit per cell memory storingdata in an upper page/lower page format, two data latches will be neededwhile programming the lower page. More generally, for memories storingmultiple pages, all of the latches will be needed only when programmingthe highest page. This leaves the other latches available for cacheoperations. Further, even while writing the highest page, as the variousstates are removed from the verify phase of the write operation, latcheswill free up. Specifically, once only the highest state remains to beverified, only a single latch is needed for verification purposes andthe others may be used for cache operations.

The following discussion will be based on a four state memory storingtwo-bits per cell and having two latches for data on each bit line andone additional latch for quick pass write, as described in U.S. patentapplication entitled “Use of Data Latches in Multi-Phase Programming ofNon-Volatile Memories” filed concurrently with the present applicationthat was incorporated above. The operations of writing the lower page,or erasing, or doing a post erase soft program are basically a binaryoperation and have one of the data latches free, which can use it tocache data. Similarly, where doing an upper page or full sequence write,once all but the highest level has verified, only a single state needsto verify and the memory can free up a latch that can be used to cachedata. An example of how this can be used is that when programming onepage, such as in a copy operation, a read of another page that sharesthe same set of data latches, such as another word line on the same setof bit lines, can be slipped in during the verify phase of the write.The address can then be switched to the page being written, allowing thewrite process to pick up where it left off without having to restart.While the write continues, the data cached during the interpolated readcan be toggled out, checked or modified and transferred back to bepresent for writing back in once the earlier write operation completes.This sort cache operation allows the toggling out and modification ofthe second page of data to be hidden behind the programming of the firstpage.

As a first example, a cache program operation for a two-bit memoryoperating in single page (lower page/upper page format) program mode.FIG. 13 is a simplified version of FIG. 10 that shows some specificelements that are relevant to the present discussion in a two-bitembodiment, the other elements being suppressed to simplify thediscussion. These include data latch DL0 434-0, which is connected DataI/O line 231, data latch DL1 434-1, connected to common processor 500 byline 423, data latch DL2 434-2, commonly connected with the other datalatches by line 435, and sense amp data latch DLS 214, which isconnected to common processor 500 by line 422. The various elements ofFIG. 13 are labeled according to their disposition during theprogramming of the lower page. The latch DL2 434-2 is used for the lowerverify (VL) in quick pass write mode, as is described in U.S. patentapplication entitled “Use of Data Latches in Multi-Phase Programming ofNon-Volatile Memories” filed concurrently with the present application;the inclusion of the register, and of using quick pass write when it isincluded, are optional, but the exemplary embodiment will include thisregister.

The programming of the lower page can include the following steps:

-   -   (1) The process begins by resetting data latches DL0 434-0 the        default value “1”. This convention is used to simplify partial        page programming as cells in a selected row that are not to be        programmed will be program inhibited.    -   (2) Program data is supplied to DL0 434-0 along I/O line 231.    -   (3) The program data will be transferred to DL1 434-1 and DL2        434-2 (if this latch is included and quick pass write is        implemented).    -   (4) Once the program data is transferred to DL1 434-1, data        latch DL0 434-0 can be reset to “1” and, during program time,        the next data page can be loaded to DL0 434-0 along I/O line        231, allowing the caching of a second page while a first page is        being written.    -   (5) Once the first page is loaded into DL1 434-1, programming        can begin. DL1 434-1 data is used for lockout of the cell from        further programming. DL2 434-2 data is used for the lower verify        lockout that governs the transition to the second phase of quick        pass write, as described in U.S. patent application entitled        “Use of Data Latches in Multi-Phase Programming of Non-Volatile        Memories” filed concurrently with the present application.    -   (6) Once programming begins, after a programming pulse, the        result of the lower verify is used to update DL2 434-2; the        result of the higher verify is used to update DL1 434-1. (This        discussion is based on the “conventional” coding, where the        lower page programming is to the A state. This, and other        codings are discussed further in U.S. patent applications        entitled “Use of Data Latches in Multi-Phase Programming of        Non-Volatile Memories” filed concurrently with the present        application and entitled “Non-Volatile Memory and Method with        Power-Saving Read and Program-Verify Operations”, filed Mar.        16, 2005. The extension of the present discussion to other        codings follows readily.)    -   (7) In determining of whether programming is complete, only the        DL1 434-1 registers of the cells of row (or appropriate physical        unit of program) are checked.

Once the lower page is written, the upper page can be programmed. FIG.14 shows the same elements as FIG. 13, but indicates the latchassignment for upper page program where the lower page data is read in.(The description again uses conventional coding, so that the programmingof the upper page is to the B and C states.) The programming of theupper page can include the following steps:

-   -   (1) Once the lower page finishes programming, the upper page (or        next page) write will begin with a signal from the state machine        controller where the (unexecuted) cache program commands are        kept.    -   (2) The program data will be transferred from DL0 434-0 (where        it was loaded into in step (3) during lower page write) to DL1        434-1 and DL2 434-2.    -   (3) The lower page data will be read in from the array and        placed into DL0 434-0.    -   (4) DL1 434-1 and DL2 434-2 are again respectively used for the        verify high and verify low lockout data. Latch DL0 434-0        (holding the lower page data) is checked as program reference        data, but is not updated with the verify results.    -   (5) As part of verifying the B state, after sensing at the lower        verify VBL, the data will be updated in DL2 434-2 accordingly,        with DL1 434-1 data being updated with the high verify VBH        results. Similarly, the C verify will have corresponding        commands to update latches DL2 434-2 and DL1 434-1 with the        respective VCL and VCH results.    -   (6) Once the B data is completed, then the lower page data (held        in DL0 434-0 for reference) is not needed as only the verify for        the C state needs to be performed. DL0 434-0 is reset to “1” and        another page of program data can be loaded in from I/O line 231        and cached in latch DL0 434-0. The common processor 500 can set        an indication that that only the C state is to be verified.    -   (7) In determining of whether upper page programming is        completed, for the B state, both of latches DL1 434-1 and DL0        434-0 are checked. Once the cells being programmed to the B        state and only the C state is being verified, only the latch DL1        434-1 data needs to be checked to see if there are any bits not        programmed.

Note that under this arrangement, in step 6, the latch DL0 434-0 is nolonger required and can be used to cache data for the next programmingoperation. Additionally, in embodiments using quick pass write, once thesecond, slowly programming phase is entered, the latch DL2 434-2 couldalso be made available for caching data, although, in practice, it isoften the case that this is only available in this way for a fairlyshort time period that does not justify the additional overhead that isoften required to implement this feature.

FIG. 15 can be used to illustrate many of the aspects of cache programin the single page mode that has been described in the last fewparagraphs. FIG. 15 shows the relative timing of what events areoccurring internally to the memory (the lower “True Busy” line) and asseen from external to the memory (the upper “Cache Busy” line).

At time t_(o) the lower page to be programmed onto the selected wordline (WLn) is loaded into the memory. This assumes the first lower pageof data has not been previously cached, as it will be for the subsequentpages. At time t₁ the lower page is finished loading and the memorybegins to write it. Since this is equivalent to a binary operation atthis point, only the state A needs to be verified (“pvfyA”) and the datalatch DL0 434-0 is available to receive the next page of data, heretaken as the upper pages to be programmed into WLn, at time t₂, which isconsequently cached in latch DL0 434-0 during the programming of thelower page. The upper page finishes loading at time t₃ and can beprogrammed as soon as the lower page finishes at t₄. Under thisarrangement, although all of the data (lower and upper page) to bewritten into physical unit of programming (here, word line WLn), thememory must wait from time t₃ to time t₄ before the upper page data canbe written, unlike the full sequence embodiment described below.

The programming of the upper page begins at time t₄, where initiallyonly the B state is verified (“pvfyB”), the C state being added at t₅(“pvfyB/C”). Once the B state is no longer being verified at t₆, onlythe C state needs to be verified (“pvfyC”) and the latch DL0 434-0 isfreed up. This allows the next data set to be cached while the upperpage finishes programming.

As noted, according to the single page algorithm with cache program, asshown in FIG. 15, even though the upper page data may be available attime t₃, the memory will wait until time t₄ before starting to writethis data. In a conversion to a full sequence program operation, such asis developed more fully in U.S. patent application Ser. No. 11/013,125,once the upper page is available the upper and lower page data can beprogrammed concurrently.

The algorithm for cache program in full sequence (low to fullconversion) write begins with lower page program as above. Consequently,steps (1)-(4) are as for the lower page process in single page programmode:

-   -   (1) The process begins by resetting data latches DL0 434-0 the        default value “1”. This convention is used to simplify partial        page programming as cells in a selected row that are not to be        programmed will be program inhibited.    -   (2) Program data is supplied to DL0 434-0 along I/O line 231.    -   (3) The program data will be transferred to DL1 434-1 and DL2        434-2 (if this latch is included and quick pass write is        implemented).    -   (4) Once the program data is transferred to DL1 434-1, data        latch DL0 434-0 can be reset to “1” and, during program time,        the next data page can be loaded to DL0 434-0 along I/0 line        231, allowing the caching of a second page while a first page is        being written.

Once the second page of data is loaded, if correspond to the upper ofthe lower page being written and the lower page is not yet finishedprogramming, the conversion to full sequence write can be implemented.This discussion focuses on the use of the data latches in such analgorithm, with many of the other details being developed more full inco-pending, commonly assigned U.S. patent application Ser. No.11/013,125.

-   -   (5) After the upper page data is loaded into latch DL0 434-0, a        judgment will be done in the address block to check if the 2        pages are on the same word line and the same block, with one        page is the lower page and one is upper page. If so, then the        program state machine will trigger a lower page program to full        sequence program conversion if this is allowed. After any        pending verify is complete, the transition is then effected.    -   (6) Some operation parameters will be typically be changed when        the program sequence changed from lower page to full sequence.        In the exemplary embodiment these include:    -   (i) Maximum program loop for the number of pulse verify cycles        will be changed from that of the lower page algorithm to that of        the full sequence if the lower page data has not been locked        out, but the number of program loops completed will not be reset        by the conversion.        -   (ii) As shown in FIG. 16, the programming waveform starts            with the value VPGM_L used in the lower page programming            process. If the programming waveform has progressed to where            it exceeds the beginning value VPGM_U used in the upper page            process, at conversion to full sequence, the staircase will            drop back down to VPGM_u prior to continuing up the            staircase.        -   (iii) The parameters determining the step size and maximum            value of the program pulse are not changed.    -   (7) A full sequence read of the current state of the memory        cells should be performed to guarantee the right data will be        programmed for multi-level coding. This ensures that states that        may have formerly locked out in the lower page programming, but        which require further programming to take account of their upper        page data, are not program inhibited when the full sequence        begins.    -   (8) If quick pass write is activated, the data of latch DL2        434-2 will be updated as well to reflect the upper page program        data, since this was formerly based on the lower verify for only        the A state.    -   (9) The programming then resumes with the multi-level, full        sequence program algorithm. If the program waveform in the lower        page process has increased beyond the upper page starting level,        the waveform is stepped back to this level at conversion time,        as shown in FIG. 16.

FIG. 17 is a schematic representation of the relative times involved inthe lower page to full sequence conversion write process. Up until timet₃, the process is as described above for the process in FIG. 15. At t₃the upper page of data has been loaded and the transition is made to thefull sequence algorithm the verification process is switched to includethe B states with the A states. Once all of the A states lock out, theverify process switches to checking for the B and C states at time t₄.Once the B states have verified at t₅, only the C state needs to bechecked and a register can be freed up to load the next data to beprogrammed, such as the lower page on the next word line (WL_(n+1)) asindicated on the Cache Busy line. At time t₆ this next data set has beencached and one the programming of the C data for the previous setconcludes at t₇, this next data set begins programming. Additionally,while the (here) lower page on word line WL_(n+1) is programming, thenext data (such as the corresponding upper page data) can be loaded intothe open latch DL0 434-0.

During the full sequence write, a status report is implemented in a waythat gives lower page and upper page status independently. At the end ofthe program sequence, if there are unfinished bits, a scan of physicalpage can be performed. A first scan can check latch DL0 434-0 forunfinished upper page data, a second scan can check DL1 434-1 forunfinished lower page data. Since, the verification of the B state willchange both DL0 434-0 and DL1 434-1 data, an A state verification shouldbe performed in the way that DL1 434-1 data “0” will be changed to “1”if the bit's threshold value is higher than the A verify level. Thispost verify will check on whether any under programmed B levels arepassing at the A level; if they are passing at the A level, then theerror is only on upper page and not on lower page; if they are notpassing at the A level, then both lower and upper pages have error.

If the cache program algorithm is used, after the A and B data areprogrammed, the C state will be transferred to latch DL1 434-1 to finishprogramming. In this case, the scan of latch is not necessary for lowerpage, because the lower page will have already passed program withoutany failed bits.

Another set of exemplary embodiments of the present invention relate topage copy operations, where a data set is relocated from one location toanother. Various aspects of data relocation operations are described inU.S. patent applications Ser. No. 10/846,289, filed May 13, 2004; Ser.No. 11/022,462, Dec. 21, 2004; and Ser. No. 10/915,039, filed Aug. 9,2004; and U.S. Pat. No. 6,266,273, which are all hereby incorporated byreference, which are all hereby incorporated by reference. When data iscopied from one location to another, the data is often toggled out to bechecked (for error, for example), updated (such as updating a header),or both (such correcting detected error). Such transfers are also toconsolidate date in garbage collection operations. A principal aspect ofthe present invention allows for a data read to an open register to beinterpolated during the verify phase of a write operation, with thiscached data then being transferred out of the memory device as the writeoperation continues, allowing the time for toggling the data out to hidebehind the write operation.

The following presents two exemplary embodiments of a cache page copyoperation. In both cases, an implementation that uses a quick pass writeimplementation is described. FIG. 18 indicates the disposition of theexemplary arrangement of latches as the process progresses.

The first version of cache page copy will write to a lower page and caninclude the following steps, where read addresses are labeled M, M+1, .. . , and write addresses are labeled N, N+1, . . . :

-   -   (1) The page to be copied (“page M”) is read into latch DL1        434-1. This can be either an upper or lower page of data    -   (2) Page M is then transferred into DL0 434-0.    -   (3) The data in DL0 434-0 is then toggle out and modified, after        which it is transferred back into the latch.    -   (4) The program sequence can then begin. After data to be        written into the lower page N is transferred to DL1 434-1 and        DL2 434-2, the latch DL0 434-0 is ready for cache data. This        lower page will be programmed. For this embodiment, the program        state machine will stop here.    -   (5) The next page to be copied is then read into DL0 434-0.        Programming can then resume. The state machine, stopped at the        end of step (4), will restart the program sequence from the        beginning.    -   (6) Programming continues until the lower page finishes.

The copy destination page address will determine whether a write is to alower or an upper page. If the program address is an upper page address,then the programming sequence will not be stopped until the programmingfinishes and the read of step (5) will be executed after the write iscomplete.

In a second cache page copy method, the program/verify process can bepaused to insert a read operation and then restart the write operation,picking up at the point where it left off. The data that was read duringthis interleaved sensing operation can then be toggled out while theresumed write operation continues on. Also, this second process allowsfor the page copy mechanism to be used in an upper page or full sequencewrite process once only the C state is being verified and one latch oneach bit line opens up. The second cache page copy operation begins withthe same first three steps as in the first case, but then differs. Itcan include the following steps:

-   -   (1) The page to be copied (“page M”) is read into latch DL1        434-1. This can be either a lower or upper page    -   (2) The data from page M is then transferred into DL0 434-0. (As        before, N, etc. will denote a write address, M, etc., for a read        address.)    -   (3) The data in DL0 434-0 is then toggled out, modified, and        then transferred back to the latch.    -   (4) The state machine program will go to an infinite wait state        until the command a read command is entered and then a read of        another page, say the next page M+1, to latch DL0 434-0 will        begin.    -   (5) Once the read of step (4) is complete, the address is        switched back to word line and block address to program the data        in steps (1-3) into page N (here, a lower page) and the        programming is resumed.    -   (6) After the read of page M+1 is finished, the data can be        toggled out, modified, and returned. Once the process is        complete, the write can be converted to a full sequence        operation if the two pages are the corresponding upper and lower        pages on the same WL.    -   (7) Once the A and B levels are done in the full sequence write,        the data in DL0 434-0 will be transferred to DL1 434-1, as in        the normal cache program described earlier, and a read command        for another page (e.g., page M+2) can be issued. If there is not        a single page to full sequence conversion, the lower page will        finish the writing and then the upper page will start. After the        B level state is done completely, the same DL0 434-0 to DL1        434-1 data transfer will occur, and the state machine will go        into state of waiting for the read command for page M+2.    -   (8) Once the read command arrives, the address is switched to        the read address and the next page (page M+2) is read out.    -   (9) Once the read is complete, the address will be switched back        to previous upper page address (program address N+1) until the        write finishes.

As noted above, the exemplary embodiments include the latch DL2 434-2used for the lower verify of the quick pass write technique in additionto the latches DL0 434-0 and DL1 434-1 used in holding the (here, 2bits) of data that can be programmed into each of the memory cells. Oncethe lower verify is passed, the latch DL2 434-2 may also be freed up andused to cache data, although this is not done in the exemplaryembodiments.

FIGS. 19A and 19B illustrate the relative timing of the second cachepage copy method, where FIG. 19B illustrates the algorithm with the fullsequence write conversion and FIG. 19A illustrates the algorithmwithout. (Both FIGS. 19A and 19B are composed of two parts, the first,upper part beginning at the broken vertical line A, corresponding to t₀,and ending with the broken vertical line B, corresponding to t₅; thesecond, lower part is a continuation of the upper portion and beginswith the broken vertical line B, corresponding to t₅. In both cases theline B at time t₅ is same in the upper portion as in the lower portion,being just a seam in two parts allowing it to be displayed on twolines.)

FIG. 19A shows a process that starts with reading of a first page (pageM) that is taken to be a lower page in this example, assumes no data haspreviously been cached, and operates in single page mode, waiting untilthe lower page has finished writing before beginning to write the upperpage. The process starts at time t_(o) with a read of the page M (Sensepage M (L)), which here is a lower that is sensed by a read at the A andC levels in this coding. At time at time t₁ the read is complete andpage M can be toggled out and checked or modified. Beginning at time t₂a next page (here page M+1, the upper page corresponding to the samephysical as lower page M) is sensed by reading at the B level, a processthat finishes at time t₃. At this point, the first page (originatingfrom Page M) (lower) is ready to be programmed back into the memory atpage N and the data read from page M+1 is being held in a latch and canbe transferred out to be modified/checked. Both of these processes canstart at the same time, here t₃. Using the typical time values describedabove, the data from page M+1 has been toggled out and modified by timet₄; however, for the embodiment not implementing a full sequenceconversion, the memory will wait until page N finishes at time t₅ tobegin writing the second read page of data (originating from Page M+1)into page N+1.

As page N+1 is an upper page, its write begins initially with averification at the B level, the C level being added at t₆. Once thestorage elements having a target state B all lock out (or the maximumcount is reached) at time t₇, the B state verification is dropped. Asdescribed above, according to several principal aspects of the presentinvention, this allows a data latch to be freed up, an ongoing writeoperation is suspended, a reading operation (at a different address thanthe suspended program/verify operation) is interposed, the write thenresumes where it left off, and the data sensed the interposed writeoperation can be toggled out while the resumed write operation runs on.

At time t₇ the interposed write operation is performed for the, here,lower page M+2. This sensing is finished at time t₈ and the write ofpage N+1 picks back up and the data from page M+2 is concurrentlytoggled out and modified. In this example, page N+1 finishes programmingat time t₉ before page M+2 is finished at time t₁₀. At time t₁₀ a writeof the data originating from page M+2 could begin; however, in thisembodiment, instead a read of page M+3 is first executed, allowing forthis page's data to be toggled out and the modification to be hiddenbehind the writing of the data originating from page M+2 into page N+2,beginning at time t₁₁. The process then continues on as in the earlierparts of the diagram, but with the page numbers shifted, with time t₁₁,corresponding to time t₃, time t₁₂ corresponding to time t₄, and so onuntil the copy process is stopped.

FIG. 19B again shows a process that starts with reading of a lower page,page M that is taken to be a lower page, and assumes no data haspreviously been cached. FIG. 19B differs from FIG. 19A by implementing aconversion to full sequence write at time t₄. This roughly speeds up theprocess by the time (t₅-t₄) of FIG. 19A. At time t₄ (=t₅ in FIG. 19A),the various changes related to the full sequence conversion areimplemented as described previously. Otherwise, the process is similarto that of FIG. 19A, including those aspects of the present inventionfound between times t₇ and t₁₂.

In both the page copy processes and the other techniques described herethat involve writing data, which states are verified at a given time canbe selected intelligently, along the lines describe in U.S. patentpublication number US-2004-0109362-A1, which is hereby incorporated byreference. For example, in the full sequence write, the write processcan begin verifying only the A level. After ever A verify, it is checkedto see whether any bits have passed. If so, the B level can be added tothe verify phase. The A level verify will be removed after all storageunits with it as their target values verify (or except a maximum countbased on a settable parameter). Similarly, after the verifications atthe B level, a verify of the C level can be added, with the B levelverify being removed after all storage units with it as their targetvalues verify (or except a maximum count based on a settable parameter).

Caching Operations in Data Latches During Program Operations

Programming operation with background data caching for other operationsis described with respect to a preferred multi-state coding.

Exemplary Preferred “LM” Coding for a 4-state Memory

FIGS. 20A-20E illustrate the programming and reading of the 4-statememory encoded with a 2-bit logical code (“LM” code). This code providesfault-tolerance and alleviates the neighboring cell coupling due to theYupin Effect. FIG. 20A illustrates threshold voltage distributions ofthe 4-state memory array when each memory cell stores two bits of datausing the LM code. The LM coding differs from the conventional Gray codein that the upper and lower bits are reversed for states “A” and “C”.The “LM” code has been disclosed in U.S. Pat. No. 6,657,891 and isadvantageous in reducing the field-effect coupling between adjacentfloating gates by avoiding program operations that require a largechange in charges. As will be seen in FIG. 20B and 20C, each programmingoperation results in moderate change in the charges in the chargestorage unit as evident from the moderate change in the thresholdvoltages V_(T).

The coding is designed such that the 2 bits, lower and upper, may beprogrammed and read separately. When programming the lower bit, thethreshold level of the cell either remains in the unprogrammed region oris moved to a “lower middle” region of the threshold window. Whenprogramming the upper bit, the threshold level in either of these tworegions is further advanced to a slightly higher level not more than onequarter of the threshold window.

FIG. 20B illustrates the lower page programming in an existing, 2-roundprogramming scheme using the LM code. The fault-tolerant LM codeessentially avoids any upper page programming to transit through anyintermediate states. Thus, the first round lower page programming hasthe logical state (1, 1) transits to some intermediate state (x, 0) asrepresented by programming the “unprogrammed” memory state “U” to an“intermediate” state designated by (x, 0) with a programmed thresholdvoltage among a broad distribution that is greater than D_(A) but lessthan D_(C). During programming, the intermediate state is verifiedrelative a demarcation DV_(A).

FIG. 20C illustrates the upper page programming in an existing, 2-roundprogramming scheme using the LM code. In the second round of programmingthe upper page bit to “0”, if the lower page bit is at “1”, the logicalstate (1, 1) transits to (0, 1) as represented by programming the“unprogrammed” memory state “U” to “A”. During programming to “A”, theverifying is relative to the DV_(A). If the lower page bit is at “0”,the logical state (0, 0) is obtained by programming from the“intermediate” state to “B”. The program verifying is relative to ademarcation DV_(B). Similarly, if the upper page is to remain at “1”,while the lower page has been programmed to “0”, it will require atransition from the “intermediate” state to (1, 0) as represented byprogramming the “intermediate” state to “C”. The program verifying isrelative to a demarcation DV_(C). Since the upper page programming onlyinvolves programming to the next adjacent memory state, no large amountof charges is altered from one round to another. The lower pageprogramming from “U” to a rough “intermediate” state is designed to savetime.

In the preferred embodiment, “Quick Pass Write” programming techniquementioned in an earlier section is implemented. For example in FIG. 20C,initially the program-verify (“pvfyA_(L)”) is with respect to DV_(AL)which is set at margin lower than DV_(A). Once the cell isprogram-verified at DV_(AL), subsequent programming will be at a finerstep and program-verify (pvfyA) will be with respect to DV_(A). Thus anadditional transitional state A_(LOW) must be latched during theprogramming operation to indicate that the cell has beenprogram-verified to D_(AL). Similarly, if QPW is implemented forprogramming to the “B” state, there will be an additional transitionalstate B_(LOW) to latch. The program verifying for B_(LOW) will berelative to the demarcation DV_(BL) and the program verifying for “B”will be relative to the demarcation DV_(B). When in the A_(LOW) orB_(LOW) state, the programming for the memory cell in question will beswitched to a slower (i.e. finer) mode by suitable biasing of the bitline voltage or by modifying the programming pulses. In this way, largerprogramming steps can be used initially for rapid convergence withoutthe danger of overshooting the target state. “QPW” programming algorithmhas been disclosed in U.S. patent application Ser. No. 11/323,596, filedDec. 29, 2005 and entitled, “Methods for Improved Program-VerifyOperations in Non-Volatile Memories,” the entire disclosure of which ishereby incorporated herein by reference.

FIG. 20D illustrates the read operation that is required to discern thelower bit of the 4-state memory encoded with the LM code. The decodingwill depend on whether the upper page has been programmed or not. If theupper page has been programmed, reading the lower page will require oneread pass of readB relative to the demarcation threshold voltage D_(B).On the other hand, if the upper page has not yet been programmed, thelower page is programmed to the “intermediate” state (FIG. 20B), andreadB will cause error. Rather, reading the lower page will require oneread pass of readA relative to the demarcation threshold voltage D_(A).In order to distinguish the two cases, a flag (“LM” flag) is written inthe upper page (usually in an overhead or system area) when the upperpage is being programmed. During a read, it will first assume that theupper page has been programmed and therefore a readB operation will beperformed. If the LM flag is read, then the assumption is correct andthe read operation is done. On the other hand, if the first read did notyield a flag, it will indicate that the upper page has not beenprogrammed and therefore the lower page would have to be read by a readAoperation.

FIG. 20E illustrates the read operation that is required to discern theupper bit of the 4-state memory encoded with the LM code. As is clearfrom the figure, the upper page read will require a 2-pass read of readAand readC, respectively relative to the demarcation threshold voltagesD_(A) and D_(C). Similarly, the decoding of upper page can also beconfused by the “intermediate” state if the upper page is not yetprogrammed. Once again the LM flag will indicate whether the upper pagehas been programmed or not. If the upper page is not programmed, theread data will be reset to “1” indicating the upper page data is notprogrammed.

Latch Utilization During Program Operation with the LM Code and QPW

As shown in FIG. 10, each bit line allows a read/write module to accessa given memory cell along a selected row of the memory array. There is apage of p read/write modules operating in parallel on a page of memorycells in a row. Each read/write module comprises a sense amplifier 212-1and data latches 430-1 coupled to a common processor 500. The senseamplifier 212-1 senses a conduction current of the memory cell via thebit line. The data is processed by the common processor 500 and storedin the data latches 430-1. Data exchange external to the memory array iseffected by the I/O bus 231 coupled to the data latches (see FIGS. 13and 14). In a preferred architecture, the page is formed by a contiguousrun of p memory cells along a row sharing the same word lines andaccessible by p contiguous bit lines of the memory array. In analternate architecture, the page is formed by either even or odd memorycells along a row. The data latches 430-1 are implemented with a minimumof n latches, sufficient to perform the various required memoryoperations, from DL1 to DLn. FIGS. 13 and 14 illustrate a preferredconfiguration for a 4-state memory where there are three latches,DL0-DL2.

Next Page Program Data loading During Current Page Programming

FIG. 21 is a schematic timing diagram for a lower page programming,illustrating background operation of loading a next page of program datainto unused data latches. The activities of the host, the I/O bus, thedata latches and memory core are shown contemporaneously. The lower pageprogramming in the LM code is illustrated in FIG. 20B where the erasedor unprogrammed state (1,1) is programmed to a “Lower Middle” orintermediate state (X,0). In this case, one bit, viz., the lower bit,will be sufficient to distinguish between the unprogrammed “1” statefrom the intermediate “0” state. For example, DL2 (see FIGS. 13 and 14)can be used to store the lower bit.

When an Nth page of data is to be written, the host initially issues awrite command to the memory for writing the page of data to a specifiedaddress. This is followed by sending the page of data to be programmedto the memory. The program data are toggled through the I/O bus andlatched into DL2 of each read/write module. Thus the I/O bus istemporary busy during this toggling-in period, which for example may beof duration 300 μs.

The lower page programming is binary and need only distinguish betweenthe “U” state from the “intermediate state” as demarcated by the DV_(A)threshold level (see FIG. 20B). Each programming pulse applied to theword line is followed by a read back or program-verify to determine ifthe cell has reached the target state representing the program data. Inthis case the program-verify is (“pvfyA”) with respect to DV_(A). Thusonly one latch from each read/write module is required to store one bitfor each cell.

With regard to the data latches, DL2 containing the program data isactively being used for the current lower bit programming operationwhich is taking place in the memory array or the memory core. Thus, thenumber of latches in use by the core is one while the other two latches,namely DL0 and DL1 remain idle.

While programming at the core continues, the two idle latches and thefree I/O bus can be used for setting up a next page of program data. Thehost can issue another command to write the (N+1)th page of data andtoggle the data via the I/O bus to be latched in one of the two freelatches, say DL0. In this way, once the core is done programming the Nthpage, it can commence with programming of the (N+1)th page withouthaving to wait another 300 μs to have the data toggled in.

At this point, two latches (e.g., DL2 and DL0) have been used, one forthe on- going programming of the Nth page (lower page) and one forcaching the (N+1)th page of program data. Thus, there is one more latchfree, but utilization of it will depend on whether the already cached(N+1)th page is an upper page or a lower page.

If the (N+1)th page is an upper page, typically belonging to the samepage cells or word line, the last free latch must, in a preferredembodiment, be reserved for optimizing subsequent programming of theupper page. This is because the implementation of “Quick Pass Write”(“QPW”) programming algorithm (mentioned in an earlier section) requiresan additional latch to store a flag to indicate if the cell has beenprogrammed close to the target state.

If the (N+1)th page is another lower page belonging to another page ofcells or word line, then the last free latch can optionally be used tocache another (N+2)th (lower or upper) page data if presented by thehost.

FIG. 22 is a table showing the number of states that needs to be trackedduring various phases of a 4-state upper page or full sequenceprogramming employing QWP. The upper page or full sequence programmingin the LM code is illustrated in FIG. 20C where some of the lower pagestates “U” or (1,1) and the “intermediate” state (X,0) are respectivelyfurther programmed to states “A” or (0,1), “B” or (0,0) and “C” or(1,0). In particular, the state “A” is programmed from “U” and thestates “B” and “C” are programmed from “intermediate”. With the QWPtechnique implemented for state “A” and “B” but not “C”, the programminginitially needs to distinguish between the basic states “A”, “B”, and“C” plus “A_(LOW)” and “B_(LOW)”, which amounts to a total of fivestates. With three bit in three latches, there are 2³ or nine possiblecodes which are more than adequate to distinguish between those sixstates.

Several Phases During Programming May Arise as Programming Progresses:

“A” Done—after all cells in the page targeted for “A” state have beenprogram-verified with respect to the D_(A) demarcation. This wouldentail having first completed the program-verified with respect to theD_(AL) demarcation. There are four states “L” (Program Lockout),“B_(L)”, “B” and “C” to keep track of. This would require two latchesstoring two bits with a predefined coding provided by a code tabletwo-bit 2CT(“A”).

“B” Done—after all cells in the page targeted for “B” state have beenprogram-verified with respect to the D_(B) demarcation. This wouldentail having first completed the program-verified with respect to theD_(BL) demarcation. There are four states “L”, “A_(L)”, “A” and “C” tokeep track of This would require two latches storing two bits with apredefined coding provided by a two-bit code table 2CT(“B”).

“C” Done—after all cells in the page targeted for “C” state have beenprogram-verified with respect to the D_(C) demarcation. There are fivestates “L”, “A_(L)”, “A”, “B_(L)” and “B” to keep track of. This wouldrequire three latches storing three bits with a predefined codingprovided by a three-bit code table 3CT(“C”).

“A”+“B” Done—after all cells in the page targeted for “A” state and “B”state have been program-verified respectively to the D_(A) demarcationand D_(B) demarcation. There are two states “L” and “C” to keep trackof. This would require one latch storing one bit with a predefinedcoding provided by a one-bit code table 1CT(“A”+“B”).

“A”+“C” Done—after all cells in the page targeted for “A” state and “C”state have been program-verified respectively to the D_(A) demarcationand D_(C) demarcation. There are three states “L”, “B_(L)” and “B” tokeep track of. This would require two latches storing two bits with apredefined coding provided by a two-bit code table 2CT(“A”+“C”).

“B”+“C” Done—after all cells in the page targeted for “B” state and “C”state have been program-verified respectively to the D_(B) demarcationand D_(C) demarcation. There are three states “L”, “A_(L)” and “A” tokeep track of. This would require two latches storing two bits with apredefined coding provided by a two-bit code table 2CT(“B”+“C”).

“A”+“B”+“C” Done—after all cells in the page targeted for “A” state, “B”state and “C” state have been program-verified respectively to the D_(A)demarcation, D_(B) demarcation and D_(C) demarcation. All targetedstates of the page have been program-verified and the programming forthe page is completed. No latch will be needed.

FIG. 23 is a schematic timing diagram for an upper page or full sequenceprogramming, illustrating background operation of loading a next page ofprogram data into unused data latches. The activities of the host, theI/O bus, the data latches and memory core are shown contemporaneously.

When an Nth page of upper page data is to be written, reference must bemade to a previously programmed lower page data. The previouslyprogrammed lower page is already latched in DL2 of each read/writemodule. With the Nth page of upper page data, the host initially issuesa write command to the memory for writing the page of data to aspecified address. This is followed by sending the page of data to beprogrammed to the memory. The program data are toggled through the I/Obus and latched into DL0 of each read/write module. Thus the I/O bus istemporary busy during this toggling-in period, which for example may beof duration 300 μs.

The upper page or full sequence programming is multi-state with thestates “A”, “B” and “C” demarcated by the D_(A), D_(B) and D_(C)respectively (see FIG. 20C). Each programming pulse applied to the wordline is followed by a read back or program-verify to determine if thecell has reached the target state representing the program data.

As shown in FIG. 22, the number of latches required during programmingvaries as to what phase the programming has proceeded to. For example,initially all three latches are employed. When all the “A” states havebeen program-verified (“A” Done”) only two latches (e.g., DL2 and DL1)are required by the memory core during subsequent programming to storefour possible states. This leave one latch (e.g., DL0) free for cacheoperation.

While programming at the core continues, the free latch and the free I/Obus can be used for setting up a next page of program data. The host canissue another command to write the (N+1)th page of data (lower pagedata) and toggle the data via the I/O bus to be latched in the freelatch DL0. In this way, once the core is done programming the Nth page,it can commence with programming of the (N+1)th page without having towait another 300 μs to have the data toggled in. The same considerationapplies to other programming phases where there is at least one freelatch as shown in FIG. 22.

Another possibility is when the programming enters a phase that onlyrequires one latch to operate and thus has two free latches for cacheoperation. For example, as shown in FIG. 22, this happens when both “A”and “B” states have been program-verified. At this point, two latchesare available. If one is already used up for loading (N+1) lower pagedata, then the remaining one may be used to load (N+2) Upper or lowerpage data.

If the (N+1)th page is an upper page, typically belonging to the samepage cells or word line, the last free latch must, in a preferredembodiment, be reserved for optimizing subsequent programming of theupper page. This is because the implementation of “Quick Pass Write”(“QPW”) programming algorithm (mentioned in an earlier section) requiresan additional latch to store one or two flags to indicate if the cellhas been programmed close to the target state.

If the (N+1)th page is another lower page belonging to another page ofcells or word line, then the last free latch can optionally be used tocache another (N+2)th (lower or upper) page data if presented by thehost.

According to one aspect of the invention, when the multiple phases of awrite operation vary as to the number of states to track, aphase-dependent coding enables efficient utilization of the availabledata latches, thereby allowing a maximum of surplus latches forbackground cache operations.

FIG. 24 is a flowchart illustrating latch operations contemporaneouswith a current multi-phase memory operation, according to a generalembodiment of the invention.

STEP 600: Beginning to operate a memory having a memory array withaddressable pages of memory cells.

STEP 610: Providing for each memory cell of an addressed page a set ofdata latches having capacity for latching a predetermined number ofbits.

Current Multi-Phase Memory Operation in Memory Array

STEP 620: Performing a current memory operation on the memory array,said memory operation having one or more phases, each phase beingassociated with a predetermined set of operating states.

Freeing Up Latches With Efficient Phase-Dependent Coding

STEP 622: Providing a phase-dependent coding for each phase so that forat least some of the phases, their set of operating states are codedwith substantially a minimum of bits so as to efficiently utilize theset of data latches and to free up a subset of free data latches.

Contemporaneous Latch Operation

STEP 624: Contemporaneously with the current memory operation,performing operations on the subset of free data latches with datarelated to one or more subsequent memory operations on the memory array.

Read Interrupt During Current Programming

FIG. 25 is a schematic timing diagram for a lower page programming,illustrating a read interrupt operation using available latches. Theactivities of the host, the I/O bus, the data latches and memory coreare shown contemporaneously.

When an Nth page of data is to be written, the host initially issues awrite command to the memory for writing the page of data to a specifiedaddress. This is followed by sending the page of data to be programmedto the memory. The program data are toggled through the I/O bus andlatched into DL2 of each read/write module (see FIGS. 13 and 14). Thusthe I/O bus is temporary busy during this toggling-in period, which forexample may be of duration 300 μs.

The lower page programming is binary and need only distinguish betweenthe “U” state from the “intermediate state” as demarcated by the D_(A)threshold level (see FIG. 20A). Each programming pulse applied to theword line is followed by a read back or program-verify to determine ifthe cell has reached the target state representing the program data. Inthis case the program-verify is (“pvfyA”) with respect to D_(A). Thusonly one latch from each read/write module is required to store one bitfor each cell.

With regard to the data latches, DL2 containing the program data isactively being used for the current lower bit programming operationwhich is taking place in the memory array or the memory core. Thus, thenumber of latches in used by the core is one while the other twolatches, namely DL0 and DL1 remain idle.

While programming at the core continues, the two idle latches and thefree I/O bus can be used for a read operation. A read operation requiressensing in the memory core (i.e., memory array) itself which is alreadypreoccupied with the current programming operation. However, the actualsensing phase of a read operation is typically much shorter than aprogram operation (typically one-tenth of the programming time) that thelatter can be interrupted to insert a sensing operation withoutincurring much penalty for performance. After the sensing, the read dataare latched in one or more of the free data latches. A user can thentoggle out the read data to the I/O bus. It is here that time can besaved since it is taking place at the same time as the program operationin the memory array.

Thus, while the lower page is being programmed, the host can issue aread command to interrupt the programming while saving the programmingstates in the data latches at the instance of the pause. Another page ofdata is sensed and latched in one of the two free latches, say DL0. Thenthe programming can resume with the saved programming states. The readdata in the data latches can be toggled out to the I/O bus while thememory array is still occupied with the resumed programming.

As described earlier, in the example of a four-state (2-bit) memory, thepreferred number of latches for each memory cell of the page is three.Only one latch to store the lower page program data is required for thelower page programming. This leaves two free latches. Only one freelatch is needed in a typically read operation to latch the sensed databit. In a preferred Look-ahead (“LA”) read operation, two free latchesare need. This will be described in more details in a later section.

FIG. 26 is a schematic timing diagram for an upper page programming,illustrating a read interrupt operation using available latches. Theactivities of the host, the I/O bus, the data latches and memory coreare shown contemporaneously. The multi-phase programming has alreadybeen described in connection with FIG. 23, resulting in different numberfree data latches available during the different phases. For example,one data latch is free after State “A” has been program-verified and twodata latches are free after both State “A” and State “B” have beenprogram-verified.

Thus, after State “A” has been program-verified, the single free latchcould be used to latch sensed data from a conventional read. On theother hand, if both State “A” and State “B” have been program-verified,the two available latches will be able to support a LA read as explainedabove.

Management of Multiple Cached Commands

Concurrent memory operations need to be managed in order to supportcache operation where one memory operation is under execution in thememory's core while data for additional pending memory operations arebeing cached at the data latches or being transferred via the I/O bus.Conventional memory devices typically do not have sufficient number offree data latches to perform cache operations. Even if they do, thepending memory operation whose data are being cached is executed onlyafter the current memory operation has completed.

FIG. 27 illustrates the package of information associated with a typicalmemory operation. When a memory is requested to perform a memoryoperation, it receives a pre-command signifying the start of a specifiedmemory operation. This is followed by the address in the memory arraywhere the operation is to take place. In the case of an erase operation,the address is the block of memory cells to be erased. In the case of aprogram or read operation, the address is the page of memory cells to beoperated on. If the operation specified is a program operation, programdata will be supplied for loading into the data latches. When theprogram data is in place, an execute-command will be issued to executethe program operation with respect to the available program data. If theoperation specified is a read operation, there will be no data sent tothe memory. The execute-command will be issued to execute the readoperation. The page of addressed memory cells will be sensed and thesensed data will be latched in the data latches for eventual togglingout via the I/O bus.

FIG. 28 illustrates a conventional memory system that supports simplecache operations. The memory system includes a memory controller 8controlling a memory chip 301 via a memory controller 8. The memory chiphas a memory array 100 controlled by an on-chip host interface/controlcircuitry 310. The control circuitry includes a state machine whichmanages the basic memory operations of the memory array. A host 6engages the memory system via the memory controller 8 which performshigher level memory functions such as mapping and maintenance.

A status signal, Ready/Busy* allows the host or the memory controller torequest a memory operation when the memory chip is not busy. Therequested memory operation is held in a buffer 322 and released to thestate machine 312 for execution when the state machine is not executinganother memory operation. For example, the memory operation MEM OPO isbeing executed in the memory array as controlled by the state machine.If there are free data latches available, the controller will besignaled to allow a pending memory operation MEM OP1 to be sent to thememory chip and buffered in the buffer 322. At the same time dataassociated with MEM OP1 will be toggled into the memory chip and latchedinto the data latches. As soon as MEM OP0 has completed execution, thestate machine will release the MEM OP1 in the buffer to begin itsexecution. Thus, in convention memory systems, a pending memoryoperation is executed after the current one is completed.

In the example shown in FIG. 28, each command must wait until the lastone is completed before it can begin execution, although its data isbeing cached during the execution of the last one. Thus, while MEM OP0is executing in the memory core, Datal associated with MEM OP1 is beinglatched. MEM OP1 will act on the cached Datal after MEM OP0 iscompleted. Similarly, while MEM OP1 is executing in the memory core,Data2 associated with MEM OP2 is being latched. This scheme forestallsthe possibility of loading both lower and upper logical pages of thesame word line and efficiently programming multi-bits in the sameprogramming operation.

There are two factors affecting the performance of program operation,particularly for sequential programming. The first relates to the timeto load the program data. As the flash memory capacity becomes larger,their page size also increases with every new generation. The largerpage of data to be programmed therefore takes longer to load into thedata latches. In order to increase the program performance, it isdesirable to hide the data loading time elsewhere. This is accomplishedby caching as much program data as possible in the background while aprogram operation is busy with the memory core in the foreground but hasits data latches and I/O bus idle.

One feature of the invention is to address the first factor by loadingmore pages into the data latches in the background during programming sothat as soon as data latches are available, they are used for cachingthe pending program data. This includes allowing data associated withmore than one command to be cached in the background during the sameforeground operation.

The second factor for program performance relates to the time to programa page, particularly for programming the page of multi-bit cells withthe same word line. As described before, a page of multi-bit cells canbe treated as a collection of individual single-bit pages. For example,a 2-bit page can be programmed and read as two somewhat independentsingle-bit pages, namely a lower-bit page and an upper-bit page. Inparticular, the lower-bit page can be programmed as soon as the programdata for the lower-bit page is available. The upper-bit page isprogrammed to the same page of memory cells in a second pass and theprogramming depends on the value of the lower page already programmed inthe cells. In this way, the two bits can be programmed in two separatepasses at two different times. However, a more efficient and moreaccurate way (with less program disturb) is to program the two bits in asingle pass in what is known as “all-bit” or “full-sequence”programming. This is only possible if all the data bits are availableduring the programming. Thus, it is preferable in practice to performall-bit programming if all the bits are available. On the other hand, ifonly the lower page data is available, the lower page will first beprogrammed. Later if the upper page data belonging to the same word linebecome available, the cells of the page will be programmed in a secondpass. Alternatively, if the upper page data becomes available before thecompletion of the lower page programming, it would be desirable to ceasethe lower page programming and instead convert to perform the all-bitprogramming.

The scheme shown in FIG. 28 would not support queuing more than onecommand in the background and therefore not support caching more thanone page of data. Furthermore, it cannot handle the situation when alower page programming is terminated prematurely and instead convertedto the execution of a different, “all-bit” programming when all the bitsbecome available.

Another feature of the invention is to address the second factor byallowing all the bits necessary for all-bit programming to be cached sothat all-bit programming can take place. Furthermore, a command queuemanager manages multiple pending commands and allows certain commands toterminate before completion in favor of the next pending command,depending on the status of their associated data.

The two features of the invention work together to enhance the programperformance by having more program data cached and allowing moreefficient programming algorithm to be employed.

According to one aspect of the invention, a current memory operation maybe under execution while other multiple pending memory operations arequeued. Furthermore, when certain conditions are satisfied, some ofthese commands for individual operations are mergeable into a combinedoperation. In one case, when conditions are satisfied to merge one ormore of the multiple pending memory operations in the queue with thecurrent memory operation under execution, the current memory operationis terminated and replaced by the operation of the merged operations. Inanother case, when conditions are satisfied to merge two or more of themultiple pending memory operations in the queue, the operation of themerged operations will commence after the current operation underexecution has completed.

One example is in programming a multi-bit page of memory cells sharing acommon word line. Each of the multi-bits may be considered as formingthe bit of a binary logical page. In this way, a page of 2-bit memorycells will have a lower logical page and an upper logical page. A pageof 3-bit memory cells will have in addition a middle logical page. Eachbinary logical page can be programmed separately. Thus, for 2-bit memorycells, the lower logical page can be programmed in a first pass and theupper logical page can be programmed in a second pass. Alternately andmore efficiently, if the program data for the 2 bits are available, themulti-bit page is preferably programmed in a single pass.

Several scenarios are possible with multiple binary programming or amerged and single-pass multi-bit programming depending on how many bitsof program data is available. Ideally, if all the bits are availablebefore programming, the multi-bit page of memory cells is programmed ina single pass. As described earlier, if only the lower logical pageprogram data is available, single-bit programming of the lower logicalpage can commence. Subsequently, when the upper logical page programdata is available, the same page of memory cells can be programmed in asecond pass. Another possibility is that the upper page data becomesavailable before the completion of the lower page programming. In thatcase, to take advantage of the more efficient single-pass multi-bit or“full sequence” programming, the lower page programming is terminatedand replaced by the multi-bit programming. It is as if the programmingfor the lower logical page and the upper page are merged or combined.

For memories with multi-bit cells, the pages of logical program datasent by a host could be a mixture of lower, upper or some otherintermediate logical pages. Thus, generally it is desirable to cache asmany pages of program data as the data latches would allow. This willincrease the likelihood of merging the logical pages belong to the samepage of memory cells so as to perform multi-bit programming.

FIG. 29 is a flow diagram illustrating the queuing and possible mergingof multiple memory operations. The algorithm for managing multiplememory operations is applied to a memory having a core array and datalatches for latching data associated with an addressed page of thearray.

STEP 710: Providing a first-in-first-out queue for ordering incomingmemory operations to be executed in the core array.

STEP 720: Accepting an incoming memory operation into the queue wheneverthe data latches are available for caching the data of the incomingmemory operation.

STEP 730: Determining if the executing memory operation in the corearray can potentially merge with any of the memory operations in thequeue. If they are potentially mergeable, proceeding to STEP 740,otherwise proceeding to STEP 750.

(By “potentially mergeable”, it is meant that at least two logical pagesassociated with the same page of memory cells can be programmed togetherin a single pass. For example, the two operations respectively toprogram a lower logical page and to program an upper logical page in amemory with 2-bit memory cells are potentially mergeable. Similarly, ina memory with 3-bit memory cells, the operations to program a lowerlogical page and an intermediate page are potentially mergeable. Also,the program operations for lower, intermediate and upper logical pagesare potentially mergeable. Returning to the 2-bit cell example, if alower logical page is under execution in the core array, it ispotentially mergeable with the next program operation pending from thequeue if the next program is to program the upper logical page belongingto the same page of memory cells. On the other hand, if an upper page isunder execution in the core array, it is not potentially mergeable,since the next pending page to be programmed will have to come from adifferent page of memory cells. Similar considerations apply to when thememory operation is a read operation.)

STEP 740: Whenever the next one or more memory operations from the queueare mergeable with the memory operation in the core array,

-   -   terminating the execution of the memory operation in the core        and begin executing instead the merged memory operations;        Else    -   Waiting until the completion of the memory operation in the core        before executing the next memory operation from the queue.        Proceeding to STEP 720.    -   (By “mergeable” it is meant that the condition for mergeability        is satisfied. In this case, the program data for both the lower        and upper logical pages are available after they have been        latched in the data latches. Similarly, “merged memory        operations” would correspond to programming or sensing both        lower and upper logical pages together.)

STEP 750: Waiting until the completion of the memory operation in thecore; and

-   -   whenever the next two or more memory operations from the queue        are mergeable, executing the merged memory operations in the        core array;        Else

executing the next memory operation from the queue in the core array.Proceeding to STEP 720.

The management of the multiple commands is accomplished by the provisionof a memory operation queue controlled by a memory operation queuemanager. The memory operation queue manager is preferably implemented asa module in the state machine that controls the execution of a memoryoperation in the memory array.

FIG. 30 illustrates a schematic block diagram of a preferred on-chipcontrol circuitry incorporating a memory operation queue and a memoryoperation queue manager. The on-chip control circuitry 310′ includes afinite state machine 312′ that serves to control the basic operations ofthe memory array 100 (see also FIG. 28.) A memory operation queue 330 isimplemented by a First-In-First-Out stack memory to hold any incomingmemory operation requests. Typically, memory operation requests areissued from the host or the memory controller (see FIG. 28.)

A memory operation queue manager 332 is implemented as a module in thestate machine 312′ in order to manage a plurality of pending andexecuting memory operations. The queue manager 332 basically schedulespending memory operations in the queue 330 to be released into the statemachine 312′ for execution.

When a memory operation such as MEM OP0 is released from the queue intoa program register 324 of the state machine, MEM OP0 will be executed onthe memory array as controlled by the state machine. At any time, thestate machine is aware of the number of free data latches available andthis status is communicated to the host/memory controller via the signalReady/Busy*. If one or more free data latches are available, the hostwill be able to request additional memory operations such as program orread. Thus, MEM OP1, MEM OP2, etc sent by the host are admitted into thequeue 330. The maximum number of memory operations in the queue will bedetermined by the number of free data latches available.

While the memory operations are pending in the queue 330, the queuemanager 332 will control the release of the pending memory operationsfrom the queue 330 to the program register 324 in the state machine.Furthermore, it determines if any of the memory operations could bemerged into a combined operation as described in connection with FIG.29. In the case where two or more operations in the queue are mergeable,the queue manager 332 will release these mergeable operations from thequeue 330 and the combined operation will be executed by the statemachine 312′ after the current operation in the state machine hascompleted execution. In the case where one or more operations in thequeue are mergeable with the operation being executed by the statemachine, the queue manager will have the state machine terminate thecurrently executing operation and execute the combined operationinstead. Thus, the memory operation manager 332 cooperates with the restof the state machine 312′ to schedule and possibly merge multiple memoryoperations.

The invention has been described using an example with a 2-bit memory.As long as data latches are freed up during a current memory operation,they can be used to cache more data for any pending memory operations.This will allow more bits of data to be loaded into the available datalatches as well as increase the likelihood of merging memory operations.Those skilled in the art will easily be able to apply the sameprinciples to memory with cells that can each store more than two bitsof data, e.g., a 3-bit or 4-bit memory. For example, in a 3-bit memory,the page of memory can be regarded as having three individual bit pages,namely lower-, middle- and upper-bit pages. These pages can beprogrammed individually at different times on the same page of memorycells. Alternatively, all three bits when available can be programmedtogether in the all-bit programming mode. This requires the cacheprogram commands to be queued for many pages. In the 2-bit memory, twoprogram commands can be executed together when full sequence conversionis possible. Similarly, in the 3-bit memory, three consecutive programcommands can be executed together when converted to all-bit or fullsequence mode. Again, the command queue manager will track which commandhas completed or terminated and which is next to execute. In this way,during programming as certain memory state milestones are reached, somedata latches are freed up and can be efficiently used for cachingpending program data.

Cache Operations During Erase—Background Read and Write Operations

The latency of an erase operation is one of the key contributors tooverall performance overhead of a flash storage system. For example, theperiod for an erase operation may be four or five times longer than thatof a program operation and ten times longer than that of a readoperation. To improve the performance of the flash memory, backgroundoperations such as cache operation become very important to make use ofthe time waiting for the erase operation to finish. The invention is tomake use of the data latches and I/O bus while the memory is busyoccupying with an erase operation in the memory core. For example, datafor the next program operation or data output from a read operation canbe performed contemporaneously with the erase operation. In this way,when the next program or read operation does take place, the data inputor output portion of that operation is already completed, therebyreducing program or read latency and increasing performance.

Erase operations can be implemented in a number of ways. One methoddisclosed in U.S. Pat. No. 5,172,338 is to erase by alternate erasepulsing followed by verifying. Once a cell has been erased verified, itis inhibited from further erase pulsing. Another erase operation,preferred for NAND memories, includes two phases. In the first phase,there is erasure by removing charges from the charge elements of thememory cells to some threshold level below a predefined “erased” or“ground” state. In the second phase, the threshold values of the erasedcells are tighten to within a well-defined threshold distribution by aseries of soft programming/verifying to the predefined “erased”threshold.

According to a general aspect of the invention, while the eraseoperation is taking place, any free data latches are used to cache datarelated to another pending memory operation.

FIG. 31 is a schematic flow diagram illustrating a cache operation inthe background during an erase operation.

STEP 760: Providing for each memory cell of an addressed page a set ofdata latches having capacity for latching a predetermined number ofbits.

STEP 770: Performing an erase operation on a designated group of pages.

STEP 780: Contemporaneously with the erase operation, performingoperations on the set of data latches with data related to one or moresubsequent memory operations on the memory array.

According to one aspect of the invention, while the erase operation istaking place, program data for a pending program operation is loadedinto the data latches via the I/O bus. In particular, during the firstphase of the erase operation when charges are being removed, all datalatches are available for caching the program data. During the secondphase of the erase operation when a soft-programming is taking place,all but one data latches are available for caching the program datasince one of the data latches is required to store a program lockoutcondition after the soft programming has verified successfully at thatlocation. If the memory architecture supports 2 bits per cell, there areat least 2 data latches, one for each bit. In the preferred embodiment,an additional data latch is used for storing certain conditions arisingduring the operation. Thus, depending on memory architecture, for a2-bit cell, there are at least two and preferably three data latchesprovided for each cell. All these data latches are available for cacheuse during the first phase of the erase, and all but one of these datalatches are available for cache use during the second phase of the eraseoperation. One or more pages of program data can therefore be loadedinto the available data latches depending on the erase phase and thememory architecture.

FIG. 32 is a schematic timing diagram for an erase operation on thememory array, illustrating a program data loading operation during thefirst, erase phase of the erase operation. The activities of the host,the I/O bus, the data latches and memory core are showncontemporaneously. As shown in the diagram, the erase operation at thememory core includes a first, erasing phase, followed by a second, softprogramming/verifying phase.

During the first phase of an erase operation the memory array or core ispreoccupied, but the data latches and the I/O bus are free for abackground operation. During this time, the program data can be loadedinto the data latches via the I/O bus. For example, in the preferredembodiment where there are three data latches for each cell, all theselatches are available for cache operation during the first erase phase.

For example, when an Nth page of data is to be written, the hostinitially issues a write command to the memory for writing the page ofdata to a specified address. This is followed by sending the page ofdata to be programmed to the memory. The program data are toggledthrough the I/O bus and latched into DL2 of each read/write module (seeFIGS. 13 and 14). Thus the I/O bus is temporary busy during thistoggling-in period, which for example may be of duration 300 μs. Withthree data latches available, up to three pages of program data can inprinciple be cached. For example, a lower page portion of the Nth pagemay be loaded, or both lower and upper page portions of the Nth page maybe loaded sequentially while the erase operation is on-going.

FIG. 33 is a schematic timing diagram for an erase operation on thememory array, illustrating a program data loading operation during thesoft programming/verifying phase of the erase operation. The activitiesof the host, the I/O bus, the data latches and memory core are showncontemporaneously.

During the second, soft programming/verifying phase of an eraseoperation the memory array or core is also preoccupied. However, asdescribed above, all but one of the data latches and the I/O bus arefree. Program data can be loaded into the data latches not used by theerase operation. For example, in the preferred embodiment where thereare three data latches for each cell, only one of the latches isemployed by the soft programming/verifying operation. Therefore thereare still two free latches available for cache operation.

For example, when an Nth page of data is to be written, the hostinitially issues a write command to the memory for writing the page ofdata to a specified address. This is followed by sending the page ofdata to be programmed to the memory. The program data are toggledthrough the I/O bus and latched into DL2 of each read/write module (seeFIGS. 13 and 14). Thus the I/O bus is temporary busy during thistoggling-in period, which for example may be of duration 300 μs. Withtwo data latches available, up to two pages of program data can inprinciple be cached. For example, a lower page portion of the Nth pagemay be loaded, or both lower and upper page portions of the Nth page maybe loaded sequentially while the erase operation is on-going.

In general, the maximum number of page can be loaded into the datalatches is a function of the memory architecture as well as how manyplanes/banks and how many chips/dies are being programmed in paralleland the speed of data transfer rate.

According to another aspect of the invention, while the erase operationis taking place, a read operation can be inserted and the resultant readdata in the data latches can be output during the erase operations.Preferably, the read operation is inserted in between the softprogram/verify operation without breaking the soft programming pulseitself. Once data is sensed and latched into the unused data latches,they can be output to the host system via the I/O bus when erase isongoing inside the array. This feature is ideal to hide systemoverheads, for example to perform read scrub operations and other systemmaintenance.

In prior art system, when an erase operation is interrupted, it willhave to be restarted from the beginning of the cycle. This could be verytime consuming especially in NAND memory.

The read operation can be inserted in between soft program and eraseverify pulses. As many read as the number of soft program pulses can beinserted into the erase operation. The sense time is additional time,but of short duration compare to the overall soft program/verifyoperation. The benefit comes in toggling out the read data as it istaking place in parallel with the on-going program/verify operation. Theread operation can also be used to perform background operation inmanaging internal control and data management.

One useful application for read during erase in a flash storage systemis where read scrub operations are implemented to maintain the storeddata in good condition. Portions of the memory where data have beenstored are read periodically to check if the programmed charges in thecells have shifted over time or changes in their environment. If so,they are corrected by reprogramming the cells with the proper margins.Various schemes of read scrub have been disclosed in U.S. Pat. No.7,012,835, the entire disclosure is incorporated therein by reference.Since a read scrub is a system operation extraneous to a host'soperation, it is best to hide a read scrub behind some other operationswhere the memory will be busy anyway. In this case, during an eraseoperation, read scrub operation could be inserted so that the readlatency can be hidden.

FIG. 34 is a schematic timing diagram for an erase operation on thememory array, illustrating a read operation being inserted and theresulting data output operation using available latches. The activitiesof the host, the I/O bus, the data latches and memory core are showncontemporaneously. As shown in the diagram, in the second phase of theerase operation the operation is soft programming/verifying. One or moreread operations are preferably inserted without interrupting thecompletion of any soft program pulses.

While the chip is in the second phase of the erase operation, thealgorithm for soft program/verify will execute. A status signal such asBUSY/READY* (not shown) will signals that the memory core is busy withinternal erase operation. At the same time, another status signal asCACHEBUSY/CACHEREADY* (not shown) will go from busy to ready to acceptread command input. As soon as a read command is entered,CACHEBUSY/CACHEREADY* will goes to busy to prevent another command frombeing entered. The read command will then wait until the current softprogram pulse is finished internally before being executed on anotheraddressed block in the same chip. After the read is done, the address ischanged back to the erase block previously being operated on. The softprogram/verify operation can resume on the erase block.

In the meantime, the read data in the data latches can be toggled out.The toggle out time is usually much longer than the read time. Forexample, the read time is about 25 μs while the toggle out time is about200 μs. So the benefit of inserting a read in an erase operation is tosalvage about 200 μs from the otherwise wasted time while waiting forerase to finish.

This cache read during erase can be inserted as many times as the erasetime would allow. However, too many reads could elongate the total erasetime and a balance should be struck between the time penalty on theerase operation the reads may incur and the toggling time salvaged fromthe reads. If there are still free time left during erase after one ormore inserted reads, the available data latches can be used to cache anyprogram data as described in an earlier section. If program data areloaded, the program operation can only start after the whole eraseoperation is completed. Enough free latches must be reserved for properexecution of the program operation, so in most cases other cacheoperations will not be possible after the program data are load.

FIG. 35 is a schematic flow diagram illustrating a specific cacheoperation for read scrub application in the background during an eraseoperation in STEP 780 of FIG. 31.

STEP 780 shown in FIG. 31 is further articulated as follows:

STEP 782: Pausing the erase operation to sense a designated page.

STEP 784: Resuming the erase operation after the data for the designatedpage are latched in the data latches.

STEP 786: Outputting the data for the designated page during the eraseoperation.

STEP 788: Scheduling the designated page for reprogramming if the outputdata contains errors.

The description for cache read so far has been made mostly to the secondphase of the preferred erase operation. The preferred erase operation iswhere the first phase is to erase all cells to some threshold levelbelow a predefined threshold and the second phase is to soft-program thecells to the predefined threshold. As described above, this erase schemeis preferred for flash memory with NAND structure since they require afairly accurate ground state and the memory is erased by biasing theN-well, which takes time. Thus it is preferable to perform all theerasing together before soft-programming. In other memory architectureusing the scheme of erase pulsing/verify/inhibit, caching operation isalso contemplated. For example, a read operation may be inserted duringa verify portion of the cycle.

FIG. 36 illustrates a preemptive background read during erase. This is amore preferably cache read when the read takes place just prior to theerase operation so that the erase operation need not be interrupted.This is possible if the read operation is known before the start of theerase operation. For example, the host may have a read request pendingor if the memory system has some read operation scheduled.Alternatively, an intelligent algorithm may anticipate where the nextread is likely to be and schedule such a read. Even if it turns out tobe a miss later, no severe penalty will be incurred. If it is a hit, itcan take advantage of the erase time to toggle out read data.

The two aspects of caching read data and caching program data during anerase operation can be combined to provide further flexibility tominimize overall system or memory overhead. Even with multiple planesand multi-chip data input operations, data input time might not fullyutilize the busy time incurred by an erase operation. In such cases,read operation and or program operation can also be added to take fulladvantage of the erase time.

Ccahe Operations During Read—Background Read and Write Operations

Cache read is usually implemented to save time when many pages aresequentially read out. The sensing for a page can be hidden during thetime to toggling out a previously sensed page so that the time forsensing does not incur extra waiting time for the user. A common schemewould be to sense the next page when the current page is being toggledout.

FIG. 37 illustrates schematically a typical read cache scheme. The(n−1)th page was sensed in a previous cycle and latched in the datalatches. At time t0, the (n−1)th page is being toggled out from the datalatches via the I/O bus as indicated by T(n−1). While the toggling istaking place, the nth page can be sensed and latched as indicated byS(n). At t2, the toggling of the (n−1) page is done and therefore it canbe followed by the toggling of the nth page of data from the datalatches as indicated by T(n). Similarly, as the nth page is beingtoggled out, the (n+1) page of data can be sensed and latched asindicated by S(n+1). This (n+1) page can be toggled immediately afterthe nth page is done toggling. Ideally, the data latches and the I/O busare fully engaged throughout the read caching so that any idle time isminimized.

According to one aspect of the invention, a read cache scheme isprovided for the case of multi-state memory cells with the need tominimize perturbation between the memory cells (Yupin Effect.) In apreferred implementation, an efficient read caching scheme is employedfor memory encoded with the “LM” coding and read with look-ahead (“LA”)correction. Both the “LM” coding and “LA” correction require additionallatch and bus activities besides the mere toggling of read data. Astraight application of the conventional scheme described in connectionwith FIG. 37 would not yield an optimized read caching.

With ever higher integration in semiconductor memories, the perturbationof the electric field due to the stored charges between memory cells(Yupin effect) becomes more and more appreciable when the inter-cellularspacing is shrinking. It is preferably to encode the multi-state memorycells of a memory using LM coding, to program the pages in the memory inan optimal order, and to read the programmed pages using LA correction.An improved read operation will implement optimum cache operation.

Cache Read Algorithm for LM Code

When the page to be read is multi-state, implementation of read cachehas to meet the requirements of the multi-state encoding used. Asdescribed before in connection with FIGS 20A-20E, the LM coding for amulti-state memory essentially minimizes the changes in the chargeprogrammed in a memory cell between different programming passes. Theexamples shown are for a 2-bit memory for coding four possible memorystates (e.g., “U”, “A”, “B”, “C”) in each cell as demarcated by threedifferent demarcation threshold values (e.g., D_(A), D_(B), D_(C)). Forexample in the 2-bit memory cell, programming to the lower logical pageadvances the threshold level at most slightly below the middle of thethreshold window of the cell. A subsequent upper logical pageprogramming further advances the existing threshold level by aboutanother quarter of the way. Thus, from the first lower to the secondfinal upper programming pass, the net change is at most about onequarter of the threshold window, and this will be the maximum amount ofperturbation a cell may experience from its neighbors along a wordline.

One feature of the LM coding is that each of the two bits, lower andupper bits, may be considered separately. However, the decoding of thelower-bit page will depend on whether the upper page has been programmedor not. If the upper page has been programmed, reading the lower pagewill require one read pass of readB relative to the demarcationthreshold voltage D_(B). If the upper page has not been programmed,reading the lower page will require one read pass of readA relative tothe demarcation threshold voltage D_(A). In order to distinguish the twocases, a flag (“LM” flag) is written in the upper page (usually in anoverhead or system area) when the upper page is being programmed. Duringa read of a lower-bit page, it will first assume that the upper page hasbeen programmed and therefore a readB operation will be performed. Ifthe LM flag is read, then the assumption is correct and the readoperation is completed. On the other hand, if the first read did notyield a flag, it will indicate that the upper page is not programmed andtherefore the lower page would have to be re-read with the readAoperation.

Decoding of the upper-bit page read will require operations readA andreadC, respectively relative to the demarcation threshold voltages D_(A)and D_(C). Similarly, the decoding of upper page can also be confused ifthe upper page is not yet programmed. Once again the LM flag willindicate whether the upper page has been programmed or not. If the upperpage is not programmed, the read data will be reset to “1” indicatingthe upper page data is not programmed.

When implementing cache read for memory using LM code, there is theadditional consideration of needing to check the LM flag which is savedon the same area as the data. In order for the state machine to checkthe LM flag, it will have to be output from the data latches via the I/Obus. This would require allocation of the I/O bus for outputting the LMflag in addition to the toggling of sensed data during a read operationwith caching.

FIG. 38A is a schematic timing diagram for cache reading a logical pageencoded with the LM code. The general scheme of toggling the last pagedata while sensing the current page is similar to that of theconventional read shown in FIG. 37. However, the sensing in the LM codeis complicated by potentially having to do two sensing passes with thechecking of the LM flag in between.

At time t0, the (n−1) logical page sensed in the last cycle is beingtoggled out from the data latches to the I/O bus as indicated by T(n−1).At the same time S₁(n) senses the next logical page (n). With the LMcoding, two cases need be distinguished: reading of a lower-bit logicalpage; and reading of an upper-bit logical page.

For the case of reading a lower-bit logical page, a preferred sensingwill begin with the assumption that the upper logical page has alreadybeen programmed so a first sensing S₁(n) will be at readB relative tothe demarcation threshold voltage D_(B). At t1 S₁(n) is done and willyield an LM flag. However, it can only be output at t2 after the I/O busis finished toggling the (n−1) page. After the LM flag is communicatedto the state machine, it is checked to determine if an upper pageexists. If the LM flag is set, the assumption was correct and thelower-bit page was read correctly. The page (n) data that has beenlatched is ready to be toggled out in the next cycle.

For the case of reading an upper-bit logical page, S₁(n) will stepthrough readA and readC, respectively relative to the demarcationthreshold voltages D_(A) and D_(C). The upper-bit page sensed data willbe stored in DL2 and the DL0 data latch is used for toggle out data (seeFIGS. 13 and 14.) At t2, the DL2 sensed data will be transferred to DL0.Again the LM flag will be checked after it has been outputted at the endof the toggling of the (n−1) page. If the upper page is programmed, allis fine and the sensed data (page (n)) in the latch is ready to betoggled out in the next cycle.

When reading an upper-bit logical page, if the LM flag is found to benot set, it would indicate that the upper page is not programmed. Thesensed data from S₁(n) will be reset to “1” so as to properly conformwith the LM coding. The sensed data is then ready for output. Then thefirst byte will be pre-fetched out and followed by the whole pagetoggling out at the start of the next cycle.

FIG. 38B is a schematic timing diagram for cache reading with LM code inthe special case of reading a lower-bit logical page when the upper-bitlogical page has not yet been programmed. Again, at t0, a first sensingS₁(n) is started and at t1, a LM flag is read. The LM flag is output forchecking at t2. If the LM flag is found to be not set, S₁(n) had readthe lower-bit page incorrectly at readB. A second sensing, S₂(n) willbegin at t3 to be performed at readA. However, this additional sensing(finishing at t4) can not be hidden behind the time to toggling of the(n−1) page, e.g., T(n−1), since the checking of the flag from S₁(n)before the second sensing will require access to the I/O bus and willhave to wait until the T(n−1) toggling is done.

Cache Read Algorithm with All-bit Sensing

In an alternative scheme, when the page on a wordline to be read ismulti-bits with multiple logical pages on the same physical page, allthe muti-bits can be sensed together in one sensing operation to savepower.

FIG. 39 illustrates a schematic timing diagram for cache read withall-bit sensing for a 2-bit memory. In the 2-bit case, the two bitsrepresenting the four memory states are sensed in the same operation.This would require sensing at readA, readB and readC to distinguish thefour states. In this case, the sensing will occur in every other cycle.For example, the sensing is only occurring on the odd cycles and will beskipped on the even cycles. The two logical pages obtained in onesensing will be toggled out sequentially at each cycle.

In the 3-bit case where there are eight states, e.g, “U”, “A”, “B”, “C”,“D”, “E”, “F” and “G”, the all-bit sensing will involve sensing atreadA, readB, readC, readD, readE, readF and readG to distinguish theeight states.

In general any multi-bit, less than all-bit sensing will serve to reducethe number sensing needed to read all the bits of the page and will helpin saving power. The memory operation queue and queue manager describedin connection with FIG. 30 can be used to manage all-bit sensingoperations by merging two or more binary-page sensing. The all-bitsensing scheme is applicable to memory with LM code and also to oneswith LA correction, which will be described in the next section.

Cache Read Algorithm for LM code with LA Correction

As for perturbations between memory cells on adjacent wordlines, theycan be mitigated during programming using a preferred programmingscheme. This will effectively reduce the perturbation by half. Theremaining half can also be corrected during read by using a preferred LAreading scheme.

A preferred programming scheme would have the pages associated with thewordlines programmed in an optimal sequence. For example, in the case ofbinary memory where every physical page holds a page of binary data, thepages are preferably programmed sequentially along a consistentdirection, such as from bottom to top. In this way, when a particularpage is being programmed, the pages on the lower side of it are alreadyprogrammed. Whatever perturbative effects they may have on the currentpage, they are being accounted for as the current page is beingprogram-verified in view of these perturbations. Essentially, thesequence of the programming the page should allow the current page beingprogrammed to see a minimum of changes around its environment after ithas been programmed. Thus, each programmed page is only perturbed by thepages on the upper side of it and the wordline to wordline Yupin effectis effectively reduced in half by this programming sequence.

In the case of a memory where each physical page of memory cells ismulti-state, the sequence is less straight forward. For example in a2-bit memory, each physical page associated with a wordline can beregarded as a single page of 2-bit data or two separate logical pages,lower and upper-bit of 1-bit data each. The physical page can thereforebe programmed in one pass with the two bits or in two separate passes,first with the low-bit page and then later with the upper-bit page. Wheneach physical page is to be programmed in two separate passes a modifiedoptimal sequence is possible.

FIG. 40 illustrates an example of a memory having 2-bit memory cells andwith its pages programmed in an optimal sequence so as to minimize theYupin Effect between memory cells on adjacent wordlines. For conveniencethe notation is such that the physical pages P0, P1, P2, . . . residerespectively on wordlines W0, W1, W2, . . . For a 2-bit memory, eachphysical page has two logical pages associated with it, namely lower-bitand upper-bit logical pages, each with binary data. In general aparticular logical page is given by LP(Wordline.logical_page). Forexample, the lower-bit and upper-bit pages of P0 on W0 wouldrespectively be labeled as LP(0.0) and LP(0.1), and the correspondingones on W2 would be LP(2.0) and LP(2.1).

Essentially, the programming of the logical pages will follow a sequencen so that the current page being programmed will see a minimum ofchanges around its environment after it has been programmed. In thiscase, again moving incrementally in one consistent direction from bottomto top will help to eliminate perturbation from one side. Furthermore,because each physical page may have two programming passes, as theprogramming moves up the physical pages, it will be better for thecurrent upper-bit page to be programmed after its adjacent lower-bitpages have already been programmed so that their perturbative effectswill be accounted for when programming the current upper-bit page. Thus,if programming starts from LP(0.0) then the sequence will be asearmarked by the page-programming order, 0, 1, 2, . . . n, . . . whichwould yield: LP(0.0), LP(1.0), LP(0.1), LP(2.0), LP(1.1), LP(3.0),LP(2.1), . . .

Cache Read Algorithm for LM code with LA Correction

According to one aspect of the invention, a scheme for caching read datais implemented so that even for read operation whose correction dependon data from a neighboring physical page or wordline, the data latchesand I/O bus are efficiently used to toggle out a previously read pagewhile a current page is being sensed from the memory core. Inparticular, the preferred read operation is a “look ahead” (“LA”) readand the preferred coding for the memory states is the “lower middle”(“LM”) code. When the read for a current page on a current wordline mustbe preceded by a prerequisite read of data on an adjacent wordline, theprerequisite read along with any I/O access is preemptively done in thecycle for reading a previous page so that the current read can beperformed while the previously read page is busy with the I/O access.

The LA reading scheme has been disclosed in U.S. patent application Ser.No. 11/099,049 filed on Apr. 5, 2005, entitled, “Read Operations forNon-Volatile Storage that Includes Compensation for Coupling,” whichentire disclosure is herein incorporated by reference. Read with the LA(“Look Ahead”) correction basically examines the memory statesprogrammed into the cells on an adjacent wordline and corrects anyperturbation effect they have on the memory cells being read on thecurrent wordline. If the pages have been programming according to thepreferred programming scheme described above, then the adjacent wordlinewill be from the wordline immediately above the current one. The LAcorrection scheme would require the data on the adjacent wordline to beread prior to the current page.

For example, referring to FIG. 40, if the current page (n) to be read ison WLm (e.g., WL1), then the LA read, as will be denoted by S_(LA)(n),will read the next wordline WLm+1 (e.g., WL2) first and save the dataresult in one data latch. Next, the current page will then be sensed inview of the S_(LA)(n) result, and will be denoted by S₁′(n).

As described earlier in connection with FIG. 40, in the LM code with thepreferred programming sequence, the lower page (e.g., LP(1.0) will beprogrammed to D_(B) or close to D_(B) (intermediate state). The upperpage (e.g., LP(1.1)) will be programmed only after the WLm+1 lower page(e.g., LP(2.0) is programmed. Then the lower page WL-WL Yupin effectwill be eliminated completely. Therefore, the data dependent correctionwill only be performed on the “A” and “C” states, and not on the “U” orthe “B” state.

In a preferred implementation of the LA read, a latch is used toindicate whether the LA read found the “A” or “C” state or the “U” or“B” state. In the former case, correction is needed and in the lattercase, correction is not needed. The corresponding cell in the currentread S₁(n) will be corrected accordingly by suitable adjustment of thesensing parameters, such as raising the wordline voltage during sensing.This is done for the entire current page by sensing once with adjustmentand another time without adjustment. The data for each cell of the pagewill then be selected from these two sensing according to whether thelatch indicates correction or not.

Read with LM code will need to check the LM flag before the read resultis finalized (either by a second pass read or by resetting the readdata.) LA correction needs to do the next wordline read first beforereading the current wordline. Therefore both the LM flag from the nextwordline read and the LM flag from the current wordline need to bechecked by the state machine. These two LM flags need to be output viathe I/O bus to the state machine when the I/O bus is not busyingtoggling read data.

FIG. 41 illustrates an implementation of read caching for the LM codewith LA correction according to the convention scheme shown in FIG. 37.Basically, the conventional scheme is for the sensing of the currentpage to be hidden inside the data toggle out time of the previous pagesensed. However, in this case, the current page sensing S₁′(n) on WLmmust be preceded by an additional lookahead read S_(LA)(n) on WLm+1. TheLM flags for each of these sensing must be output via the I/O bus beforethe sensed data are ascertained. The current page sensing S₁′(n) isperformed in view of the data from the S_(LA)(n) to yield the correcteddata for the current page. It will be understood that S₁′(n) may befollowed by an additional S₂′(n) if n is a lower-bit page and theupper-bit page is not yet programmed as shown in FIG. 38B.

In the next cycle beginning at t0, the corrected sensed data of page nis then toggled out as indicated by T(n). At the same time, the currentsensing has now moved to the next page with S₁′(n+1), which must bepreceded by S_(LA)(n+1). However, the output of the LM flags from thesesensing must wait until the toggling of the page n, T(n) is done.Furthermore, S₁(n+1) can only be performed after the result ofS_(LA)(n+1) is definite. Thus, S₁′(n+1) can only be performed outsidethe data toggling period and therefore cannot hide behind it. This addsan additional sensing time when the latches and I/O bus are not fullyutilized, and the wasted time is repeated for every subsequent cycles.This implementation degrades read performance for the user when the LAcorrection is used.

A preferred implementation of cache read in LM code with LA correctionis to pipeline the next wordline sensing and current wordline sensing insuch a way that all the sensing will be hidden inside the data toggle.The next wordline sensing is always executed ahead of the currentwordline sensing. Inside each group of data toggle, the current wordlinesensing will be executed and followed by the next-next wordline sensing.When the group of data has finished toggle out and the I/O bus isavailable, the next-next wordline LM flag will be fetched out first andchecked. If the LM flag is in the state indicating the upper page notprogrammed, then the next-next wordline sensed data will be reset to “1”(for no correction). The current wordline LM flag will be checkedsubsequently. Depending on the current wordline LM flag, either thesensed data is kept or another sensing need to be executed (in the caseof lower page read) or the data will be reset to all “1” (in the case ofupper page read). All these sensing and data toggle out can be managedwith 3 data latches for a memory with 2-bit memory cells.

FIG. 42 illustrates an improved read caching scheme with the LM code andLA correction. The first cycle from −t5 to t0 is when the current page(n) on WLm is read and is different from the rest of the cycles. Asbefore, the LA correction require a preceding read of S_(LA)(n) wherereadA, readB and readC will sense the cell states on WLm+1. The LM flagfrom this read F_(LA)(n) will be output at −t4 and checked. If the flagindicates the upper page is not programmed on WLm+1, the data beingsensed will be reset to all “1”, indicating that there will be nocorrection. If the flag indicates the upper page is programmed, then thelatched data indicating correction or not will be kept as it is. At −t3,the current page on WLm will be sensed with S₁′(n) and possibly S₂′(n)in accordance with the LM code and LA correction scheme describedearlier. In contrast to the scheme illustrated in FIG. 41, a preemptivelookahead read is also performed for the next page (n+1). Thus, at time−t2, S_(LA)(n+1) is performed and at −t1, its LM flag is output andchecked.

After the first cycle, at the beginning of the next cycle at t0, thepreviously sensed data from S₁′(n), now LA corrected, will be toggledout as indicated by T(n). The page address will be incremented first to(n+1) which reside on a wordline given by the order indicated in FIG.38. Thus, at time t0, with the start of T(n), sensing of the (n+1) page,S₁′(n+1) can begin right away since its prerequisite lookaheadS_(LA)(n+1) has already been completed in the previous cycle. At the endof S₁′(n+1) at t1, the LM flag F(n+1) will be fetched out and checkedand any additional action will follow depending on the LM flag. Thecorrected page (n+1) data will then be ready for toggling in the nextcycle. In the meantime, while the page (n) is still being toggled out,the lookahead sensing S_(LA)(n+2) for the next page can be performed inadvance and within the toggling period of T(n).

As soon as T(n), the toggling for page (n), is completed, the next cyclewill start and T(n+1) follows with the toggling out of the LA correctedpage (n+1) data. The cycle for page (n+1) continues in similar manner asthat for page (n). The important feature is that the lookahead read fora given page is preemptively performed in an earlier cycle.

FIG. 43 is a schematic flow diagram illustrating the improved readcaching:

STEP 810: In each reading cycle where a page from a series thereof is tobe sensed from a memory, outputting a previous page sensed in the lastcycle in a current cycle.

STEP 830: Sensing a current page during said outputting the previouspage, said sensing the current page being performed on a currentwordline and requiring a prerequisite sensing at an adjacent wordline soas to correct for any perturbation effect from data on the adjacentwordline.

STEP 850: Preemptively performing said prerequisite sensing of theadjacent wordline related to the current page in a cycle earlier thanthe current cycle.

FIG. 44 is a schematic flow diagram illustrating a further articulationof STEP 850 of FIG. 41:

STEP 852: Outputting a first flag obtained as part of the data from saidprerequisite sensing.

STEP 854: Adjusting the data from said prerequisite sensing according tothe output first flag.

STEP 856: Latching the data to indicate whether corrections need to bemade for said sensing of the current page to follow.

FIG. 45 is a schematic flow diagram illustrating a further articulationof STEP 830 of FIG. 41:

STEP 832: Performing said sensing of the current page with and withoutthe correction from the prerequisite sensing.

STEP 834: Outputting a second flag obtained as part of the data fromsaid current sensing.

STEP 836: Responsive to the second flag, revising the data from saidcurrent sensing either by leaving the data unchanged, or adjusting thedata a predetermined value, or obtaining new data by repeating saidsensing of the current page under another set of sensing conditions.

STEP 838: Latching either the corrected or uncorrected revised dataaccording to whether the data from the prerequisite sensing indicatecorrection or no correction.

The above algorithm has been described using the 2-bit LM code. Thealgorithm is equally applicable LM codes for 3 bits or more.

Although the various aspects of the present invention have beendescribed with respect to certain embodiments, it is understood that theinvention is entitled to protection within the full scope of theappended claims.

1. A method of reading a non-volatile memory having addressable pages ofmemory cells on associated wordlines, comprising: sensing and latching apage of data in each of a series of reading cycles, wherein said sensingand latching in a current reading cycle are directed to a current pageof data on a current wordline and are performed responsive toprerequisite data from an adjacent wordline so as to correct for anyperturbation effects therefrom; preemptively sensing and latching theprerequisite data for the current page prior to the current readingcycle; and performing in the current reading cycle, other than the firstreading cycle, said sensing and latching of the current page whileoutputting a previous page sensed and latched in the just past readingcycle.
 2. The method of claim 1, wherein the memory cells each storesone bit of data.
 3. The method of claim 1, wherein the memory cells eachstores more than one bit of data and each page of memory cells on awordline is associated with more than one page of binary datacorresponding to the individual bits of each memory cell.
 4. The methodof claim 3, wherein said sensing and latching of the prerequisite datafor the current page further comprises: outputting a first flag obtainedas part of the prerequisite data from said prerequisite sensing;adjusting the prerequisite data responsive to the first flag; andlatching the adjusted prerequisite data to indicate whether correctionsneed to be made for the current page.
 5. The method of claim 4, whereinthe first flag indicates whether the prerequisite page of memory cellson the adjacent wordline has been programmed with all the bits.
 6. Themethod of claim 5, wherein: when the first flag indicates that theadjacent wordline has not been programmed with all the bits, adjustingthe prerequisite data to indicate no correction is needed.
 7. The methodof claim 5, wherein: when the first flag indicates that the adjacentwordline has been programmed with all the bits, adjusting theprerequisite data to indicate that correction is needed.
 8. The methodof claim 3, wherein said sensing and latching of a current page furthercomprises: performing said sensing of the current page with correctionsfrom the adjusted prerequisite data; outputting a second flag obtainedas part of the current page; responsive to the second flag, refreshingthe current page either by leaving the page unchanged, or adjusting thepage to a predetermined value, or replacing with a new page obtained byrepeating said sensing of the current page under another set of sensingconditions; and latching the refreshed current page for outputting inthe next reading cycle.
 9. The method of claim 8, wherein said sensingof the current page for those memory cells that needs correction asindicated by the adjusted prerequisite data is performed at a wordlinevoltage with a predetermined boost.
 10. The method of claim 8, whereinthe second flag indicates whether the current page of memory cells onthe current wordline has been programmed with all the bits.
 11. Themethod of claim 10, wherein: when the second flag indicates that theadjacent wordline has been programmed with all the bits, said refreshingthe current page is a null operation by leaving the current pageunchanged.
 12. The method of claim 10, wherein: the memory cells eachstores two bits of data; the page of memory cells is constituted from alower-bit page and an upper-bit page; and when the second flag indicatesthat the adjacent wordline has not been programmed with all the bit, andthe current page being sensed is an upper-bit page, said refreshing thecurrent page is by resetting the current page all to a predetermined bitindicating the upper-bit page has not been programmed.
 13. The method ofclaim 10, wherein: the memory cells each stores two bits of data; thepage of memory cells is constituted from a lower-bit page and anupper-bit page; and when the second flag indicates that the adjacentwordline has not been programmed with all the bits, and the current pagebeing sensed is a lower-bit page, said refreshing the current page is bysensing the current page relative to another predetermined reference.14. A method of reading a non-volatile memory having addressable pagesof memory cells on associated wordlines, comprising: sensing a group oftwo binary pages together and latching the two binary pages separately;outputting a latched binary page of data every operating cycle exceptthe first two cycles; and wherein said sensing and latching areperformed every two operating cycles during said outputting of a latchedbinary page from a previous operating cycle.